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programers* program evaluators* field testers* occasional special 
consultants* and conferees* can be numbered by the scores. Through 
them the front line activities of development and dissemination of 
programed material was .accomplished. 
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concerned for the success of the project and for translating its re* 
stOts into ever improved teaching procedures. 
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INTRODUCTION 

The Great Lakes Colleges Association Programed Instruction 
Project had its origin in an authorization of the Board of Director i 
of the Great Lakes Colleges Association dated January, 1962. The 
board approved formulating a proposal for a project dealing w th 
programed instructional materials at the college level. To that 
end a conference was called on March 23, 24, 1962, at which time 
the potential of programed instruction for colleg© teaching was 
discussed with reference to experiences already gained at Antioch, 
Earlham, and Oberlin Colleges. From this conference came the first 
draft of a proposal and a series of agreements about the nature of 
the proposed project. Dr. Daniel Smith of Earlham College, with 
the assistance of others, assumed responsibility for formulating 
the final draft of the proposal. 



Members of the conference, appointed by their presidents, 
representing all the colleges in the association except Kenyon were 
the following: 



Albion College, Dr. Paul Carnell 
Antioch College, Dr.'^ William S. Johns 
Denison University, Dr. Dewey Slough 
De Pauw University, Dr. Clark Norton 
Earlham College, Dr. Daniel Smith 
Hope College, Dr. Ralph Perry 
Kalamazoo College, Dr. Walter Waring 
Oberlin College, Dr. Celeste McCollough 
Ohio Wesleyan University, Dr. Francis Alter 



Wabash College, Dr. George Lovell 



College of Wooster, Dr. John Warner 

i 

GLCA President, Dr. Eldon Johnson 

Though GLCA submitted the proposal to a foundation in May, 
1962, it lay dormant until March 196L, at which time the Research 
Committee of the Great Lakes Colleges Association consisting of 
chairman Dr. Sdmuel Baskin, Antioch College; Dr. Paul Carnell, 
Albion I Dr. Robert De Haan, Hope College $ and Dr. Celeste 
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McCollough*' Oberlin College; decided to submit^the proposal to the ) 
U. S. Office of Education. The 0. S. Office of Educa on accepted 
the proposal for support and funded it in the sunnmiert .963 under 
Public 65-'864 Title VII t Part B, Dissemination activi..ies concerned 
with the more effective utilizations of media for educational 
purposes . 

In the summer of 1963 » the Great Lakes Colleges Association 
initiated its PROGRAMED INSTRUCTION PROJECT with supporting funds 
from the 0. S« Office of Education* Originally a two-year project » 
it was extended for one year in 1965. 

The project began with the following objectives: 

1. To determine areas within the curriculum common to the same 
department in the various colleges which may be covered 
adequately by programed self instruction; 

2. To determine specific behavioral objectives to be achieved 
in the areas under 1 above; 

3. To review existing programs to identify those which promise 
to fulfill the objectives under 2 above, and arrange for 

detailed evaluation and field testing of these programs; 

' 1 

I 

4. to develop programed learning sequences where^^er the ob- 
jectives determined in 2 above cannot be achiftved through 
programs identified in 3 ; 

; 5. To promote basic research in instruction related to objec- 

tives which have been determined in 1 and 2 above, but 
which cannot be satisfied through activities under 3 and 4; 



6* To evaluate the broader effects of programed instruction. 

i 

Dr. Robert F. De Haan, Professor of Psychology and Chairman of 
the Department at Hope College, Holland, Michigan was appointed 
Director of the Project. 



Extension of Contract to June 1966 

The attainment of some of the objectives of the original con- 
tract,' especially objective number 6, "the evaluation of the broader 
effects of programed instruction”, depended upon the construction of 
a number of programs which bi?'eause of sheer lack bf time in the first 
two years of the project were not produced. After the summer of 
1964 a large number of programs were on hand and it became possible 
to' proceed to evaluate them out in a variety cf situations, thereby 
gathering the data^and experience necessary for the attainment of 
objective number 6* 



An extension of the contract to June 1966 was negotiated in 
the Spring of 1965* The overall objective of the extension of the 
contract became that of evaluating the broader effects of programed 
instruction on the total instructional process in the liberal arts 
college setting* The objectives given below specify in greater de~ 
tail the meaning of this objective. They are as follows: 



al material prepared in 1964 in as great a variety of instructional 
situations as possible. 

» • 

2. To study the effects of evaluating programed material and 
the effects of preparing programed material on the instructional 
processes and to ascertain which of the two has the greater impact 
on instructional processes* 

3. To compare two methods of preparing programed material- 
individual and team preparation on subsequent instriictlonal process 
es, that isy do' plgogramers working individually or-aq a team have 
greater effect ciji^the instructional processes? 

4. To dippeminate the results of the project to other small 
liberal arts colleges *. 

\ Beginning and Ending Dates : June 27, 1963, to June 27, 1965. 

" Extension to June 30, 1966* 

'■ J • 

' Contractor: The Great Lakes Colleges Association, Detroit 



1. To evaluate and compare the uses of programed instruction 




Metropolitan Airport, Hichigari, Dr. Eldon L. 
Johnson, President* 
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Director : Dr. Robert F. DeHaan, Hop^ Collegj|, Holland, 



Michigan* 
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Format of the Final Report 

The first two chapters introduce the reader to the administra*' 
tion of the Project giving special emphasis to the methods for pre- 
paring programed materials* evaluating both commercial and GLCA 
produced programs* and disseminating information about programed 
instructional materials among GLCA colleges. 

The third chapter reports the results of the project in terms 
of the number of programs produced, each of which is described in 
some detail. The results are also given in^ terms of the evaluati'on 
of the deans and in terms of effeciency of preparing programed 
materials. 

The first threji- chapters deal with the attainment of the first 
four objectives of original contract. 

The fourth chapter reviews selectively the research on pro- 
gratped instruction. his chapter serves as an introduction to the 
remaining chapters ot the report. 1 

I Chapter 5 reports the. evaluation of commercial programs and 

serves specifically to meet the third objective in the original 
contract . 



The sixth chapter is perhaps the central chapter in the report 
in that it is an evaluation of GLCA produced programs. In this 
chapter the first objective of the extended contract is attained. 



Chapter 7 discusses the rationale for preparing programed 
material and shows in a theoretical way its relationship to certain 
aspects of teaching. ;'This chapter ' introduces^ the next two chapters. 
■The three chapters together report the attainment of objective six 
|of the original contract and objective two of extended contract. 

J 

Chapter 8 is a brief report of a preliminary study made of the 
jimpact of programing on teaching. Chapter 9 is a report of a pilot 
,study of the relationship between programing and teaching made under 
more controlled conditions. 



S 



Chapter 10 summarizes and concludes the report and makes re^ 

jcommendations for further development and research. 

{ 

The appendices for all the chapters are grouped in Part V. 
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Chapl|er 1 

THE FIRST YEAR 



Pr^j.minary Visi ts to QLCA Colleges 



the prlsidents^and deLs^^'^wi^^fSe^Proleo? '’anr^^h’ 

with them. lai-« “reject, and to become acquainted 

wxuii tnem, late in the summer and earlv in thA fan iqg^ 4««xui.eY 

Director made a pralimlnarv ^r^oV+ IZ ^ t tne tall of 1963, the 

Association. On each canZ, ^ in the I 

administrators exZessed ZZZ °°^‘^tally received, and college 

expressed a good deal of enthusiasm for the Project. 




in the development Z o^ovZf Z • °euld be involved 

period of timet’lor exLZZ ° ® shorter 

only six persons developing ’progrZs fZrauZhe^fe instead of 

5..2.1.; ,.;na iiSiJriiJS.Eni'S':;.;,' e~'e^" r r •“— ■■ 

the college Th« of staff teaching functions in 

ledgraiZrand cref??Z®fZ„?fdr'- " disseminating know- 

immediately obvioZ! ?L ^ • P*'°8ramed instruction were 

idents and officials decided to ’ consultation with QLCA pres- 
the Office ofEZeaUon? >^ecommend the change in plans^o 



A second decision was made in the course 

Each person preparing 
would be expected to participate in a training 
his responsibility. Rather than assume that a 
produce a program without being trained in the 
of writing programs, the Project was to provid 
training for him,- These early changes made a 
future course of the Project. 



of the preliminary 
programed material 
workshop as part of 
faculty member could 
special techniques 
e a period of) intensive 
great impact upon the 



.h.. Er''" 

number of them qualitatively under less controlled ZnSitLiZ®® 
^gainzat ioh Of • Liaison Cdmmiti-AA | 

to submit names of Persons''toZZJdodtZ\iaLd*‘cLmitted“°Jiey^“ 









k 
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were instructed to select members from the faculty and to avoid as | 
much as possible naming administrators as liaison persons. The j 
names of these persons were submitted to the Office of Education foi* 
approval as per contract. A number of the liaison persons appointed 

had served on the original committee that drafted the proposal for 
the Project. 



The members of the Liaison Committee were* 

Albion College 

Dr. Paul Carnell, Chairman, Department of Chemistry 

Dean Dillery, Department of Biology 










Antioch College 

Dr. Robert MacDowell, Associate Professor of Mathematics 

“ Professor Richard MeisJ|^dr, Department ofCphii 

Denison University * 

Dr, Irvin Wolf, Professor of Psychology 






DePauw University 

Dr# Clark Norton D^HfCtor of. Graduate Studies; Asst* Dean of the 
University, Professgi^^^ Of Political Science 

2nd year ~ Dr^t^, Kenneth Wagoner, Chairman, Department of 
Psychology 

Earl ham Colleg e 

Dr. Daniel Smith, Assistant Professor of Education, Coordinator of 
Self-Instruction 

Hope College 

Dr. Ralph Perry, Professor Romance Languages Department 
Kalamazoo College 

(Dr. Walter W. Waring, Chairman, Department of English 
[ Kenyon College 



Dr. Bruce Haywood, Dean of the College 
l Oberlin College 

|Dr. Loche Van Atta, Associate Professor of Psychology 
l Ohio Wesleyan University 



Dr. Francis Alter, Chairman, Department of Education 

IgAyear - Dr. Joseph Wetmore, Professor of Education 

j Wabash College 

Dr. Paul Mielke, Associate Professor of Mathematics 
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College of Wooster 

Dr. Donald 6. Beanes Assistant Professor of Education 
3rd year - Dr. Sam Cho, Department of Psychology 




First Liaison Committee Meeting 
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The Liaison Committee first met on Friday, October 10, 1963 
at the Indianapolis Airpott Hotel. All the colleges were repre- 
sented at this first meeting except Kenyon and Oberlin. Since 
most of the liaison members were new to the Project, the Director 
spent most of the session outlining the Project for them.^ Topics 
discussed included: the time table of the Programed Instruction 

Project, the role of the liaison person, and the Director ' a . sche- 
dule and activities. A proposed structure of initial campus con- 
ferences with faculty members for the fall of 1963 was established 
and a procedure for these conferences outlined as described below. 
This meeting also resulted in preliminary plans for the structure 
of the Winter Work Conferences for winter of 1964, definition of 
the role of the GLCA programer, and plans for the 1964 summer work- 
shop and media production schedule. 

At its first meeting, the Liaison Committee made specific 
plans and set dates for the next visit of the Director to the 
various campuses. The purposes of this visit wer| to present the 
general outlin^ 6f the Project to the faculties, parti<sularly to 
the department chairmen, and to consult with faculty membcsrs who 
might be interested in preparing programed material during the 
summer of 1964 under the aegis of the Programed Instruction Project. 
The arrangements for the initial campus conferences were made by 
the liaison persons. The liaison person^also acted as contact 
person during the visit of the Director. 




In each college the Director followed the same general plan 
of introducing the Project. The Director described the Programed 
Instruction Project at a faculty meeting. The length of time 
allotted to the presentation at the faculty meeting varied from ten 
to thirty minutes. Questions were invited at each meeting, but 
only rarely were any forthcoming. The entire day following the 



faculty meeting was then devoted to conferences between the Director 
and interested faculty members. At some colleges only one or two 




^ See Ai^peftdlx’ A-l’, foi»' Director’s* visitittg ’Schediiie 
and the h&ihber of faculty ‘cohtacts at eaeh college. 

^ Except at Earlham College where the faculty meeting 
was cancelled for administrative reasons. 
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The 

ings with 
of higher 
Project. 



Director introduced his presentatio.. at the faculty meet- 
a statement that a new assessment of the effectiveness 
education could be made through the Programed Instruction 
The Director defined and described programed instruction, 



mentioning programed textbooks, branching programs, adjunct pro- 
grams, language labs, and ’’single concept” films. He presented the 
two major thrusts of the Project; the development of new programs 
suitable for small liberal arts college s, an d the testing of exist- 
ing programs. He also sketched out theC gerTe raS' plan of operation 
for the GLCA Programed Instruction Project; Winter Work Conferences 
a programers' worksl^d|), summer production of programs, and fall 
evaluation of commerq|al programs. 




The Director 4§|)Psited with each liaison person a tiumber of 
Faculty Participati^^f Questionnaires (FPQ) and Departmental Ques- 
tiqqnaires (DQ). TH|j first of these was an instrument designed to 
entourage and assess faculty participation in tha Programed In- 
struction Project whereby faculty members could indicate the level 
of participation in the Project they would prefer for themselves. 
The DQ was designed to find out the extent to which various depart- 
ments were using programed materials. The liaison person collected 
the complete FPQs and the DQs after the Director's visit and for- 
warded them to his office. From the FPQs were drawn the names of 
faculty members who were interested in participating in the Winter 
Work Conferences as well as those who^w’ere interested in preparing 
testing, or inspecting programs during the summer of 1964.^ 



Departmental Meetings in the Colleges 



The original contract with the Office of Education called for 
departmental meetings as follows: 



Departmental leaders will conduct a series of department 
al meetings to discuss and develop the following areas; 






persons appeared for individual conferences, and at others there 
were as many as 15 to 20. The Director helped the faculty members 
to isolate areas and topics in their disciplines which might be 
programed with the support of the project. On the second day at a 
luncheon meeting the Director explained the Project to departmental 
chairmen in greater detail and allowed time for further questions. 






I 



(a) areas within the subject which might profitably be 
covered by the use of programed self-instruction 
and media through which they might be presented 
effectively; 









See Appendix A-2^ for a summary of results of FPQ, indicatin 
faculty willingness to test and inspect published programs 
and GLCA programs* 
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(b) 

(c) 



specific behavioral objectives to be achieved with- 
in these at*eas; 



existing programed materials which might achieve 
these behavioral objectives and thus be worth furthe 
testing i 



(d) 



a roster of faculty members interested in either | 
evaluating existing programs (including construction 
of tests and outlining of content) or developing ' 
new ones; and 



(e) 



a roster of faculty members interested in devoting 
time and effort to attend a subsequent conference 
to spell out in detail the recommendations of var- 
ious colleges in each subject* area. 



It became clear to 
the faculties of the 6LCA 
estimated the ini1;erest of 
ional materials a^d their 
The very terminplf»p;y such 
"identifying area* to be 
of faculty membl|];>i!| . Sine 
mental chairmen^ :ne could 
)iold departmentii meeting 



the Director on his initial contacts with 
that this part of the contract over- 
the faculty members in programed instruct-! 
sophistication in the use of such materials* 



as "specifying behavior!|| Cbjectives" and 
programed" was objectionable to a number 



e the Director had no control over depart- 
do no ni'jre than advise and urge them to 
s about programed instructional materials. 



It was also difficult to designatj\_aiLeas within a subject 
matter that could be programed. The i nteres t i* producing programed 
material seems to grow in a particular^ faulty member who is ex- 
cited about some topic or problem in his discipline which he feels 
might lend itself to being programed and has relatively little to 
do with whether or not the topic is one that is of great interest 
to a department or to a profession jn general. Desire to write 
programed material is more closely related to the teaching interests^' 
of individual faculty members than to departments. Thus the depart 
mental approach to developing interest in programed materials, al- 
though Sound for a long-term project, was not possible in a two-year 
project. It was more than could be expected of the departments to 
accomplish in two months what is outlined from (a) through (e) in 
the above paragraph in the contract. 



If the contract had been followed explicitly, it is doubtful 
whether sufficient momentum could have been generated in time for 
participants to be selected from the departments for the Winter 
Work Conferences and for the preparation of proposals to produce 
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f P*'og*'amed instructional materials during the summer of 1964.** De- 
partments were therefore bypassed in trying to recruit faculty 
ptirsons to attend the Winter Work Conferences and to prepare pro- 
gramed materinls. Participants were recruited directly by the 
liaison persons using the rpQ, The work that was delegated to the 
departmental mootings in the contract was assigned instead as a 
major objective of the Winter Work Conferences* 



College Level Programed Haterials Library 

By the end of 1963, a number of’ published programs had been 
received by the Programed Instruction Project* On January 17, 1964, 
a list of them was sent to the liaison persons to serve as a bib- * 
liography for departments and individual faculty members who were 
interested in programed Instructional materials* The number of 
programs received is included in Appendix A-3* 



Second Liaison Committee Meeting 
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The second meeting of the Liaison Committee was bold at 
Middleton House on tb«s campus of Denison University in Granville, 
Ohio, on January 10, J964* The major purposes of the meeting were 
to plan the Winter ^prk Conferences and to make preliminary plans \ 
for the summer 1964 preparation of programed materials* The Liaison 
Committee suggested that the "specif ication of behavioral objective^ 
by each department would be a desirable goal for the Winter Work 1 

Conferences* The Liaison Committee did not however, approve of ! 

the term "behavioral objectives," as used in the contract* The 
term smacked of educational jargon* In addition, the committee 
was concerned that the faculty members from the different colleges i 
would feel ccorced if they all were expected implicitly to reach j 
agreement on them* They strongly resisted such pressure* The term* 
"educational objectives" was adopted* j 
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The Winter Work Conferences | 

The objectives of the Winter Work Conferences were to: I 

II 

I 

1* Get acquainted with colleagues, compare notes on teaching^ 
programing, media* j 

i 

I 

i 



As further indication of the inability of departments to 
come to grips with issues presented in the contract, | 

Appendix A-4, outlines the number oi Departmental Question- | 
naires and the reaction of the department to programed 
instruction that were returned to the project office* ! 



3. 



4. 



2* Look over published programed materials that will be 

supplied for the conference. Decide whether to test any 
of th^' available programs in 6LCA colleges. 

Review disciplines to discover areas most amenable to 
programed instruction, discuss educational objectives 
that apply to those areas, and formulate tentative plans 
for future work in discipline. 

Review submitted proposals for preparing programed mater- 
ial in the summer of 1964 with faculty members who sub- . ^ 

mitted them. iiake suggestions and recommendations a'bout ’ 
testing materials and developing new materials. 

Invitations were extended to faculty members who had indicated 
their interest by^ means of rPQ.5 The liaison persons also submitted 
additional names, of n ersons^ to receive invitations’. The confer- i 
ences followed ,thj (sOh edul^ below: 

February 9B aind 29 - Humanities Winter Work Conference 



A. 



March 6 and 7 



March 20 and 21 



- Natural Sciences Winter Work Conference 

- Social Sciences Winter Work Conference 



The conferences were held at the Avis Hotel in Detroit Metropolitan 
Airport, and each followed a similar format. The following sche- 
dule is typical: 

GLCA - PROGRAMED INSTRUCTION PROJECT 
Social Sciences Winter Work Conference^ 

Conference Program 

Friday, March 20, 1964 

10:00 - 10:15 Greeting from GLCA, Eldon L. Johnson, 

President 



10:15 - 10:30 



Objectives of the Conference, Robert F. 
DeHaan, Director 



® Lists of Participants in the three Winter Work Conferences 
are given in Appendices a-5 to A-7. 
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For a list of resource persons used at each conference, seel 
Appendix A-8. Their task was to work with each of the de- | 
partKents to help them think through the problems of pro- | 
graming for their particular discipline. j 



I 



10:30 - 11:30 


"The Process of Programing" - Dr* George 
Geis, Center for Research on Learning and 
Teaching, University of Michigan* 


XXsSO ^ X2*30 


• 

First departmental meeting 


12:30 - 1:30 


Luncheon 


1:30 - 3:00 


"Theoretical Background to Programing In- 
struction" - Dr. Dan Smith, Director, Self- 
Instruction Project, Earlham College* 


3:00 - 3:30 


• 

Coffee Break 


3:30— 5:00 


Second departmental meeting 


5:00 - 6:00 


Fre * Tim® 


6:00 - 7:30 


Dinner 


7:30 ^ 8:30 


Group Discussion, DeHa^n, Resource Persons 


Saturday, M 4 ^t*ch 21, 


1964 



9’, 00 - 10:30 "Other Media for Self-|||Struction in the 

Social Sciences", Joan Rosengren, Project 
on Educational Communication, Teachers 
College, Columbia, Univ,, New York, Mew York 

10:30 “ 12:00 Third departmental meeting 

12:00 1:00 Luncheon 



1:00 - 3:00 


Fourth departmental meeting 


3:00 


Adjournment 



It should be noted in passing that the Director was hard 
pressed to find consultants for each discipline represented in the 
conferences* At the time of the Winter Work Conferences no one 
could be found with sufficient experience in programing in .dis- 
ciplines such as history, home economics, and art to act‘as a^ 
consultant* For several other areas, for example, speechj political 
science and music, only one person could be found. The present 
pj’oject is likely to establish a pool of persons in GLCA who will 
be expert both in their disciplines and in programing to serve as 
consultants to other colleges and college groups* 

A summary by colleges and by disciplines of participants at 
the Winter Work Conferences is given in Appendix A-9. 
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P^^’ticipants * Evaluatio?i of the Winter Work Conference$( 



In their written evaluations of the meeting participants in 
the Winter Work Conference mentioned most often the helpfulness of 
the overall view of programing. About 40 of the members made some 
reference to a broadened understanding of the usefulness of pro- 
gramed material, of its aims, and of the value of taking a new 
look at teaching and learning. An additional 10 people mentioned 
the helpfulness of specific speakers. Smith and Geis. Participants! 
also emphasized the value of the interaction with faculty persons 
from other colleges and of the exchange of ideas and information 
about teaching and programing on both departmental and interdis- 
ciplinary bases. 



Meetings of departmental discussion groups stimulated think- 
ing about programing possibilities and about evaluating teaching 
methods already being used. About 20 persons mentioned help from 
resource persons in these discussion groups. The opportunity to I 
examine available programs was stimulating to many of the partici- | 
pants, and 18 of them mentioned that this was their first opportuni- 
ty to take more than a cursory look at thi(fb material. About 3 5 ! 

persons found coiljtact with an expert in the programing field helpfull 
to their understanding of both program writing ai^d' program use in 
the classroom** llif the 20 persons who listed specific information 
on program writing, 7 went on actually to write a .program* Thus, 



• the Winter Work Conference provided many of the participants a 

much needed over\iew of the process and potentials: of programing 



Director’s Evaluation of Winter Work Conferences 



The Winter Work Conferences achieved several positive accom- 
plishments. For most of the participants this was the first GLCA 
conference, and getting acquainted with colleagues and comparing 
notes on teaching and programed media were of highest priority. 

In this process many persons also became familiar with programed 
materials available and considered applications for these mat- 
erials in their own teaching. Beyond this a major accomplishment 
of the conferences involved reviewing and completing proposals for 
the preparation of programed materials during the coming summer. 
The Director and Dr. Daniel Smith went c /er a number of proposals 
with their authors. In addition the conference stimulated several 
more faculty members to submit proposals. 



i 






Despite these successes, however, the Winter Work Conferenc 
left some goals unachieved. First, the entire problem of evaluat 
ing commercial programs was still unsolved at the end of the con- 
ference and was left to be approached more systematically at the 
May 3 Liaison Committee meeting. Moreover,, except for the'^cheml. 



economists, ' Librarians, and to some extent, for the 



stry 






eig|i-language group, departmental groups did not succeed 



/ 
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areas amenable to programed instruction nor in discussing education* 
al objectives in those areas* Appendix A-13 has Chemists* report* ! 




Selection of Summer 1964 Programers i 

Besides the workshop, another major activity during the winter of 
1964 was the gathering of proposals from faculty members of the 
Great Lakes colleges to prepare programed materials during the 
summer of 1964. This activity reached a climax during the Winter , ^ 

Work Conference dnd culminated in the awarding of 24 programing \ 

grants for the summer of 1964, subject to the approval of the re- | 
vised contract and budget of the Office of Education* ^ | 

I I 

The procedure used in obtaining proposals was as follows: An; 

announcement cf the opportunities to prepare programed materials i 
during the summer of 1964 was made at each college during the ; 

Director* s initial campus conference during the fall of 1963 and j 

early winter 1964* At the close of each visit the Director left j 

with the liaisop person a number of FPQs which were to be given to ! 

faculty members whereby they could indicate as individuals their | 
proposed level of involvement * j 

■ As mentioned earlier, the first contact with many potential 
programers wai made during the Director* s initial visits to the 
campuses* At this point he talked with a number of faculty members 
who were interested in programing and helped them prepare tentative 
proposals for writing programed materials during the summer of 1964* 

Faculty members were invited by means of the FPQ and by a ! 

follow-up application form to submit proposals for summer work ; 

grants. The application for the summer work grant to prepare pro- 
gramed materials contained the following questions: ‘ ^ 

1, What part of your discipline do you propose to program? | 

2. In what way is the content you plan to program important? » 

For example, is it important because so many students are ; 
involved in learning it? because it is so difficult to j 
learn? because there is no time to deal with it is class,* 

etc*? i 

3* What do you expect students to learn from your program? | 

What objectives have you established? Be as specific as | 

possible. I 

i 

4. Is there anything unique about your plan of procedure that; 
should be noted? 

5 

5. Describe any other features of your plan that do not fit \ 

readily into the above category of questions. | 
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If you propose to do research on programed instruction 
rather than prepare programed materials, please outline 
your purpose, hypotheses, procedures, expected outcomes. 

Sixty-one questionnaires containing proposals for preparing 
programed material were returned to the office of the Programed 
Instruction Project. After tjve_ Winter Work Conferences another 13 
proposals were received. Distribution of the proposals received 
and grants awarded by colleges’“Ts given in Appendix 10. It is notable,,,^^^^' 
that there is a rather high, positive correlation between number of’ 
proposals submitted by each college and the number of programing 
grants awarded to it. , \ 

Six of the summer programing grants were awarded earlier than j 
the rest because the persons making the proposals had to meet an* j 
early deadline for commitment for the summer *s work. The Director, \ 
in consultation with Drs. Dan Smith of Earlham College and Paul | 

Carnell of Albion College agreed on the value of these proposals and, 
accepted them for Summer Work Grants. I 

Persons subm|.tting proposals were informed thit the final award^- 
ing of grants would be made by a subcommittee of the Liaison Comm- 
ittee at the end of March, 1964. Each one who submitted a proposal 
was encouraged |d attend one of the Winter Work Conferences, to re- 
view his proposal with colleagues, revise it if necessary in the 
light of what the colleagues recommended, and then resubmit the 
proposal. A subcommittee of the Liaison Committee, consisting of . 

Drs. Dan Smith, Earlham and Locke Van Atta, Oberlin, reviewed ths ‘ 
proposals on March 27, 1964. Wo explicitly stated criteria for s 
selecting the programs were formulated. Such criteria were developed 
however, for the extension of the project into 1966. From a total ! 
of 63 proposals that were submitted, 23 grants were awarded. One j 
grant was divided into four parts and awarded to four persons, and i 
one grant was also divided among four librarians for producing one I 
total library program. Two larger grants for half time programing ‘ j 

on a year long basis were also made. ^ 

The letter to programers regarding the terms of the agreement ! 
contains the following: 1 

i 

The terras of the ^reement are as follovrs: you will work for a ^ 

period of nine weeks . duri ng the summer of 3 964 , including attendance! 
at the training workshop which will be hold in two sessions. The 1 
first session will be held during the weeik of June 15, and the se- I 
cond session will be held during the week of July 13. Your ex- f 
penses will be paid to these workshops. During the seven remaining ! 
weeks you will work according to the general specifications of your ‘ , 

proposal under the supervision of project personnel. At the end of • 
the summer we are tentatively planning to hold a two day evaluation * 
workshop. We will also wish you to attend that workshop, for | 

4 

ft 

% 









which your expenses will be paid. 



For your work you will receive a stipend of two-ninths your 
annual salary to be paid at the conclusion of the summer. Up to 
$500 may be spent on materials and clerica 1 help. 



The recipients 


of the grants and 


the initial statement of 1 


their topics to be programed are listed 


below. 


Programers 


Department 


Original Program Topic 


Albion College 


English 


"The Craft of Poetic . 


James Cook 




Criticism" 


Antioch College 






Clarence Leuba 


Psychology 


"Cognates adjunctive to Man*j 


W« M. Lotkowski’^ 


Geology 


"Procramed Unit on Soils" 


Lois Peiekoudas* 


Political 


gP.roce.d.ur!!ea of Systematic 


j 


Science 


Analysis in Political 
Science" 


Denison Univeristy 1 


Kenneth MarshalJL 


English 


"Formal Sj;ructure of the 


O'"'' 




Short Story" 


Horten Schagrin 


Philosophy of 


"The Langua|e of Science" 




Science 




DePauw University 


Preston Adams 


Botany 


"Gross and Microscopic Ana- 
tomy of the Green Plant" j 


Thomas Davis 


Mathebatics 


"Conic Section, Polar 
Coordinates, the Line" 


Frank McKenna 


Psychology 


^^onceptg of Business and In- 
dustrial Psychology" 


Fred Silander 


Economics 


"General Concept of a Theo- 
retical Model" i 


Earlham College 




I 

-r i 


Robert Brewster 


German ■ 


"Programed German Vocabulary 
through Cognates" , 


Ansell Gooding 


Geology 


"Self-instruction Materials ! 
for Geology" 


Wm. Stephenson 


Biology 


"Biochemistry for General ! 
Biology Course" | 


1 

Hope College 


Phillip Van Eyl 


Psychology 


"Techniques of Reading 
Technical Articles" • 






• • 



*Half-time grants of year-long duration. 

‘ X i 14. 



Programers 


Department 


Original Program Topic 


Kalamazoo College 


LAwrence Smith 


Music 


"Ear-training in Music" 


Kenyon College 




1 


Paul Schwartz 


Music 


"Musical Notation and Term 
inology" 


Oberlin College 


Peter Hawkins 


Chemistry 


"Selected Organic Chemistr 
Concepts" 


Forbes Whiteside 


Art 


"The Nature and Uses, of 
Color" 



Ohio Wesleyan Univeristy 

Robert Montgomery Religion 

Vant Kebker Economics 

Joseph Wetmore Education 



"The Synoptic Problem" 
"Capital Budgeting" 
"Public School Finance" 



Wabash College 

Richard Strawp French 

Wooster Collegt^^. 

Richard Knudten Sociology 



"Structure of French" 
"Social Organization" 



Library Grants 
J. McRee Elrod 
Evan Farber 

Peter Kidder 
j Marian Mullendor 



Ohio Wesleyan "Preparing a Unit Card" 

Earlham "Organization and 

Terminology" 

Kenyon "Ordering LC Cards" 

DePauw 



jFractional Grants 
1 Guy Stern 



I Wolf« Halott 
James Cope 



Paul Arnold 



"Adaptation of German | 

Textbooks" \ 

"Absolute Pitch Training" | 

"Associating Bird songs | 

with Bird Pictures" 

"Print Making" ! 



1 I 

< Third Liaison Committee Meeting 

9 I 

The Third Liaison Committee meeting was held on May 8, 1964 I 
at the Indianapolis Airport Hotel* Committee members reviewed 
activities at the colleges by means of a chart showing the number | 
of contacts, proposals received, participants at work conferences | 
and awards granted. After discussing the quarterly report to the 
Office of Education, the difficulty of defining areas for program- 
ing and of setting up behavioral objectives, and after adopting the 
principle of evaluating a few programs in depth rather than many ! 
superficially, the committee formulated plans for a series of campus 

15 _ “ 
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visits by the Director to display commercial programed materials 
thereby to recruit faculty persons to evaluate some of them. Two 
questions in evaluating commercial programed ma tler^raTTirije^ be 
asked: 1) Does programed material actually teach? and 2) how do 

you use programed material? The Director was instructed to call 
in a consultant to advise about processing evaluation data with a 
computer. 




The Liaison Committee also discussed briefly the possibility | 
of systematically evaluating RLCA produced programs at a lat.er datel 



Visits to GLCA Colleges to Display Commercial Programed Materials i 



The 

programed 


itinerary of the Director* 
materials in the spring of 


s visits 
1964 was 


to display commercial [ 
i as follows: ' 


May 


14 


Hope College 


May 


25 


j 

Oberlin College 1 


May 


IS 


Kalamazoo College 


May 


26 


1 

College of Wooster | 


May 


19 


Wabash College 


May 


27 


Ohio Wesleyan University 

I 


May 


20 


Earlham College 


May 


28 


Denison University | 


May 


21 


Antioch College 


May 


29 


1 

De Pauw University 


May 


22 


Albion College 
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A less propitious time could hardly be imagined for this visit 
to the colleges to exhibit commercial programmed materials. In 
many of the colleges comprehensive examinations and/or final exam- 
ination^ were being given. Professors were in the throes of grading 
papersT Generally speSking, few were in a mood even to walk across ; 
campus to examine the materials. Approximately 90 persons in 11 ! 

colleges inspected the programs with various degrees of seriousness!.^ 
More important than the number of persons who inspected the pro- j 
grams I however • was the number of those who expressed interest in 
testing programso Eighteen individuals made more or less firm [ 

, commitments to test a program. Fifteen of these persons did actual*- 
ly test a program during the fall and winter of 1964c of which about 
a dozen were able to report results. I 

Summer Training Workshop, 1964 

The major activities during the summer of 1964 were the pre- | 
parations of programed material and the two week training period j 
which all programers received. Conducted by the University of 
Michigan at Fairlane Estate^ Dearbornc Michigan, the training | 

1 



^ See Appendix A-11. 
— 







! centered on the process of programing. About this time it became j 
apparent that the training in programing might be effective not i 

i only in producing high quality programed material, but also in i 

making a significant impact on teaching in QLCA colleges. In 
addition, if the summer programing were to be continued for several! 

' summers, a significant pool of trained programers would be formed I 
in 6LCA colleges,^ ! 

I 

1 The workshop was divided into two sessions, the first running 

; from Sunday evening, June 14, through Friday Afternoon, June 19, 

1964 and the second session from Sunday evening, July 12, through 

* Friday afternoon, July 19, 1964. The number of Great Lakes Golleges 
; Association participants did not exceed 24, and the GLCA staff did ; 

; not exceed two. 

! " 

The first session of the workshop was devoted to training 
^ the participants in the rudiments of programing. The second sessiojn 
'was concerned vrith editing, evaluating and managing programs. Be- i 
I tween sessions the participants worked on their own campuses and 
.returned to the second session with in-progress programs. The I 
: details of the workshop program were worked out vfith the director • 

; of the workshop^' who was designated by the University, with his 

staff, and with'the director of the Programed Instruction Project.® 

* I 

I The workshop staff consisted of a Director, ^ liaison adminisl* 

.trator, and program editors furnished by the University. The 
workshop vras conducted on about a four-to-one stu^^nt-faculty ratio. 

Principles of Training 

The principles of training under which the workshop was con- 
i ducted are stated below and elaborated in Chapter 6. I 

! 

! 1. The process of programing depends upon an analysis of the 

j subject matter, and is not to be governed by preconceivedj 

ideas about the mechanics and "proper" format, e.g., 
linear, branching, small or large steps, or immediate 
reinforcement. | 

a. By analysis of subject matter is meant the formula- 
tion of ob jectives i^^n behavioral terms, determination* 

of criterion frames, and the construction of the | 

fewest number of teaching frames to bring the student 
to the behaviors required by the objectives. 

b. Pre and post-tests are integral parts of the analysis 
of the subject matter since they provide evidence of 



For a schedule of the workshop, see Appendix A-12. 
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'the student's achievement of the objectives. 



2 . 



Learning perceptual and conceptual discrimination by dis- 
covery is often more effective than is "shaping of behav- 
ior" by reinforcement. Discovery is generally initiated 
by the presentation of a problem or a panel of data about 
which questions are asked of the subject. The process of 
discovering the answer produces the desired behaviors, 
which may then be practiced with further examples or may 
be tested later in the post-test. 






a* If the subject makes errors, it is desirable first to 
ascertain whether the errors interfere with the stu- 
dent's learning and, secondly, whether the material 
is clearly presented. If material is unclear or am- 
biguous, steps are taken to rectify the sequence of 
presentation or to state the matter more precisely. 
More frames are not necessarily added just to prevent 
errors from being made. 



b. Emphasis is placed on producing meaningful, thoughtfu;. 
responses from the subject, as opposed to the almost 
m<ilaningles8 or rote responses that are frequently 
fpund to be required in some progrqfns. 

c. The results are "lean" programs rather than the redun- 
dant ones so often found on the market at the present 
time r 

3. Programs need to be validated empirically to ascertain 

whether they teach what the author claims they will teach 
in his statement of objectives. 



a. Test subjects need to be available from the earliest 
stages of program writing. 

b. Test subjects can also be used to discover whether th 
program is appropriate for the intended audience. For 
example, if test subjects can successfully do all 
criterion frames before working through the program, 
there is noneed to write the program since they 
already know all it has to teach. 

Training Personnel 

The following persons were involved in the training workshop 
at the University of Michigan: Mr. Dale M. Brethower, an editor, is 
a psychologist and assistant director of "ALLP" , a language program-* 
ing project sponsored by the Office of Education and directed by 
Professor F. Rand Morton. 



Dr. George L. Gels, director of the workshop, is a Research 
Associate in the University of Michigan's Center for Research on 
Learning and Teaching. 

! Dr. Patricia O'Conner, an editor, is a Research Associate in 

' the Center for Research on Learning and Teaching and a lecturer in 
the Department of Psychology at the University of Michigan. 

i Mr. Geary A. Rummler, editor, is Director of the Center for 

Programed Learning for Business in the Bureau of Inductrial. Rela- 
tions at the University of Michigan. 

i 

i Mr. Albert W. Schrader, III, editor, is Publications Director 

of the University of Michigan's Bureau of Industrial Relations and 
a Consultant to the Center for Programed Learning for Business. 

Dr. Donald E. P. Smith, part-time editor. Chief of Reading 
Improvement Services, Bureau of Pyschological Services and Associ- 
ate Professor of Education at the University of Michigan first came 
to the University of Michigan in 1952. 



Dr. M. Daniel Smith, an editor from GLCA, is an Assistant 
Professor in Education at Earlham College. 

Evaluation of Summer Workshops 

Participants in the Summer Training Workshops returned thought- 
ful evaluations of the sessions. Nearly everyone said it had been 
helpful or worthwhile to their program writing experience. 



- workshop was very helpful. If its main objective was to ge 
me writing frames, then that was accomplished. 

- tremendous surge of work on third and fourth days , good 
morale, team research. 






- liked the approach, 
programing. 



using programing techniques to teac^ 



- didn't realize how valuable until I returned home and ?tart 
ed to work. All GLCA people should go through this in small 
groups. 

Individual consultations with editors were of most value to parti- 
cipants, and many mentioned that more time spent in this way would 
have been helpful. 

- writing of frames in presence of experts and being able to 
check with them were most valuable parts of workshop. 








would have liked four days of concentrated work with editor- 
ial conferences in late afternoons. 

helpful was presence of editors who have--despite their 
youth--had sound practical experience in programing. 

conversations with GeiSf Stephenson^ Gooding helped me clear 
ideas of my objectives. 

progress came through the work I did, during talks with 
editors. 



Several persons suggested that the last day, Friday, made the sessioA 
too long, and several others indicated that there were too many, gen- 
eral presentations. 

- most of talks and lectures hindered my progress . . naeded to 
work alone. 

- efforts interrupted by taping session. 

- prefer lecture-discussions held at earliest or latest times 
so they would not interfere with programing woi»k. 

A half dozen persons said they would have liked more undisturbed time 
for frame writing. Most participants would have liked tnore time with 
individual editors and some kind of signup system so that everyone 
would have seen an editor ; several times. There were sharply dif- 
fering opinions on the value of lectures. 



especially helpful was Don Smith's demonstration of principle 
that discr iminat ior precedes synthesis. 



the variety of methods and iisights into learning theory were 
the most valuable things I learned. 

speech by Don Smith on reading program he wrote helped me see 
program can be tested and revised. 



- liked the Markel movies, Geis on Retention; slowed by some 
speeches . 

- lectures kept me from my program. 

- without extra lectures could have gone home on Thursday. 

Especially during the second workshop, several persons who had begun 
to make real progress in frame writing found lectures and discussion 
distracting. A few found the change of pace in lectures invigoratin, 
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and pertinent to the [broad concept of teaching* * 

The Director and \glcA consulting editor Daniel Smith felt that 



the workshops had been competently handled by the University of Mich* 
igan group, that programers were helped with their program writing i 
and their understanding of behavioral objectives for programs and | 
for other courses as well. The physical setup of the workshop was i 
a little unwieldy, and too much time was spent traveling between 1 

motel and meeting place. They agreed that another such workshop i 
should be conducted at a motel in which conferences could be held 
in individual rooms, with editors' time made available on a signup | 
basis and with each programer seeing an editor at least twice each | 
day. Any lectures like those on E6RUL-RULEG should be introduced | 

early in the program.® A brochure or a workbook which included \ 

excerpts from college level programs would be especially helpful, [ 
especially since the practicum supplied by the University staff did 
not include subject matter germane to the college level. ; 



Below is a list Qf programs produced in the summer^ 1964 ; 

ANATOMY OF THE P|,ANT by Preston Adams 

Department' of Botany and Bacteriology, DeBauw University 

GERMAN VOCABULARY LEARNING THROUGH COGNATES ny Robert Brewster 
Department of Languages, Earlham College 

POETRY: METHOD AND MEANING by James Cook 

Department of English, Albion College 

ANALYTIC GEOMETRY by Thomas A. Davis ' 

Department of Mathematics, DePauw University 

PROGRAM ON CRYSTAL STRUCTURE by Ansel ^M. Gooding 

Department of Geology and Soil Science, Earlham College 



A SELECTED INTRODUCTORY ORGANIC CHEMISTRY by Peter J. Hawkins 
Department of Chemistry, Obe.rlin College 



CAPITAL BUDGETING by Vant W. Kebker 

Department of Economics, Ohio Wesleyan University ! 

A PROGRAM TO SELF-INSTRUCTION SOCIAL ORGANIZATION by Richard Knudteii 
Department of Sociology, College of Wooster ! 



9 EGRUL-RULEG simply means presenting material by first providing 
students with examples(egs ) , from which he generalizes the rule 
(rul) or conversely, presenting rules followed by examples. 
Egrtil presentation seems most effective. * 



HUMAN NATURE by Clarence Leuba 

Department of Psychology, Antioch College 
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PROGRAMMED UNIT ON SOILS by William Lotkowski 
^ Department of Earth Sciences* Antioch College 

I 

THE FORMAL STRUCTURE OF THE SHORT STORY by Kenneth B. Marshall 
Department of English* Denison University 

SELECTION PROCEDURES by Frank S. McKenna 

Department of Psychology* DePauw University 

THE TWO-SOURCE THEORY ABOUT THE SYNOPTIC PROBLEM by Robert fJontgomery 
Department of Religion* Ohio Wesleyan University j 



PROCEDURES OF SYSTEMATIC ANALYSIS IN POLITICAL SCIENCE by Lois 
Department of Political Science* Antioch 

THE LANGUAGE OF LOGIC by Morton Schagrin 

Department of Physical Science* Denison 

MUSICAL SCORE fiip^ABING by Paul Schwartz 

Departm(||)| of Music* Kenyon College 

INTRODUCTION TO THEORY AND INTRODUCTION TO THe/ SUBJECT MATTER OF 

by Fred S. Silander* Department of Economics* ECONOMIC^ 

Pauw University 

EAR TRAINING by L. Smith* H. Ray and R. Hammar | 

Department of Music* Kalamazoo College j 

i 

BIOCHEMISTRY FOR GENERAL BIOLOGY by William K. Stephenson * 

Department of Biology* Earlham College 

HOW TO STUDY A TEXTBOOK by Phillip Van Eyl 

Department of Psychology* -Hope College 

FINANCING OUR PUBLIC SCHOOLS by Joseph N. Wetmore 

Department of Education* Ohio Wesleyan University 

DISCRIMINATION IN HUE* VALUE AND INTENSITY by Forbes Whiteside 
Department of Art* Oberlin College 




College Pelekoudas 



liversity 



A grant was also made to four librarians* each of whom was to 
prepaid® a quarter of the program. The program is designed to train 
newly acquired library personnel to become proficient in the routine 
tasks in library science that they would be required to perform. 

The programers are Mr. J. McRee Elrod* Ohio Wesleyan University; 

Miss Marian Mulendore* DePauw University; Mr. Peter Kidder* Kenyon 
College; and Mr. Evan Farber* Earlham College. 



L 
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Four fractional grants were also made for small projects. 
Professor Paul Arnold, Art Department, Oberlin College, received a 
aaall grant to produce a series of very brief films on intaglio 
printing .techniques} Professor James Cope, Biology Department, 
Earlham College, received a grant to make tape recordings of bird 
calls to be associated with colored slides of birds; Professors 
Irvin Wolf and Malott in the Psychology Department at Denison 
University received a grant to experiment with procedures for 
training absolute pitch; Professor Guy Stern, formerly of the Ger- | 
man Department of Denison University, received a grant to adapt an j 
already existing series of small German textbooks to programed j 

learning and self-instruction. * l 



Surv<ay of Use of Programed Instruction ; 

In Decenfber 1964 a brief survey was made of the use of pro- 1 
grained instructional materials during that semester in GLCA j 

colleges. A tpjjial of 7 0 programs were reported being used by j 

more than 3,00# students: Ten programs in biology, 11 in chemistry, 
four in education, five in economics, six in English, three in 
geology, 12 in mathematics, three in modern foreign languages, 
five in music, one in physical education, one in physics, eight in 
psychology, one in religion. ’ 

See Appendix A-14 for more details of the survey. ! 

I 

1 

I 

Survey of Use of Programed Instruction ! 

. _ j 

In December 1964 a brief survey was made of the use of pro- j 

grained instructional materials during that semester in GLCA colleges 
A total of 70 programs were reported being used by more than 3,000 j 
students: Ten programs in biology, 11 in chemistry, four in educa- I 
tion, five in economics, six in English, three in geology, 12 in j 
mathematics, thre§ in modern foreign languages, five in music, one 
in physical education, one in physics, eight in psychology, one in 
religion. See Appendix A-14 for more details of the survey. 
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THE SECOND AND THIRD YEARS 
Evaluation of Commercial Programs 

thA last two weeks of May, 1964, the Director visited 

GLCA colleges to display the commercial programs that he had 
accumulated during the previous nine months as described in Chapter 

interest in trying out one or two 
qn^f following Fall. The Liaison Committee in the 

should hA^i meeting had decided that the program® to be evaluated ! 
should be tested in the most rigorous manner possible. The experi- ! 
mental design included carefully matched control groups, comparisons) 

and%re-and posttesUn^'''^-'' traditional teaching techniquesl 

During the ^pmmer of 1964 twelve instructors agreed to test 
commercial prog^aj|s. Later three more instructors were found. The 

cesr«r *'®i^®f fifteen think through tlie initial pro- 

cess of evaluation during the late summer and early "fall. The 

designs might be categorized as "field experiments" in the terminol- 
gy of Cartwright and Zander ( Group Dynamics . 1960, pp. 51-52). 

The actual evaluation took place during the school year 1964-65. 

control and experimental groups and relatively 
controlled variables, each experiment employed four common 
instruments : 



1. a student evaluation list form for the experimental group 

2. a student evaluation form for the control group 

3. an instructor evaluation form 

4. a personnel data roster form 

I^®®® Pi^pvided some standardization for the evaluation. 

But the evaluations were far less adequately controlled than had 

complete report of the process and results of 
evaluating commercial programs is given in Chapter 5. 

Testing GLCA. - ' Prepared Programs 

^ Two kinds of testing are used in the preparation of programed 

instructional “aterial. The first may be called developmeSta! test- 
ing, the second field testing. 
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Developmental testing is done very early in the preparation 
of the material. As its name suggests its purpose is to test the 
program while it is being developed. It is generally done on a one** 
to-one basis, the programer presenting his program frames on cards 
one card at a time, to a student or a series of students acting as ! 
test subjects. The difficulties and misconceptions as well as 
successful comp.letion of teaching frames are noted by the programer, 
who observes the student to see where he hesitates, what misunder** \ 
standings he has, and what .is clear to him. - 

After the student has completed the segment of the prdgram . 
presented to him, the programer discusses his observations wi*th the 
student to obtain as much information as possible from the testing 
situation. The programer then reviews and rewrites his program in 
the light of the comments made by the test subjects. The programer 
may submit his program to similar on-the-spot testing many times in 
the course of developing the program. 

Field testing serves a different purpose. It is largely a 
test of the instructions for administering the program and a test 
of how well the program teaches under the conditions in which it 
will be used in the ''field." The instructor who field tests the 
program is asked to evalua;te it. The student is given a question- 
naire about his j:*esponse to the program and also makes notes on the 
program as he go^s through it. Occasionally a perceptive student 
will give as much information on a field test as previous students I 
did on a development test, put usually the information obtained is 
more general. 

Field testing which was conducted during the fall of 1964 wasi 
an integral part of producing programed material on this project. 

For each field tester the programer furnished a sufficient number ' 
of copies of the program, sufficient pre-and-post tests, and instruc- 
tions to the faculty person administering the program and to the 
students taking it. 

The Programed Instruction Project made it pcssible for the ; 
program writer to do some additional indirect assessment of the 
effectiveness of the program with instruments provided by the pro- j 
ject office. Below are listed the instruments and instructions ! 

that were supplied to each field tester to administer: * 

{ 

1. Instructor Program Evaluation Quests’ onnaire : To be com- | 
pleted by the instructor during and after the program has | 
been completed by the students. It will record your 
evaluative comments on the program,^® - ! 



10 « 

For samples of the actual instruments 
See Appendices B-1 and B-2. 
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Evaluation Ques tionnaire ; To be given to students 
alter they have completed the program to obtain their re« 
action and e valuative comments on the program.^® 

^ ^tadent Perso nnel Data Roster Form ; To be used to collect 

such data as the students' scholastic aptitude test scores, 
their rank in their graduating class in high school, etc. 

4. Time Sheets: (optional) to be filled in by students at the 
conclusion of their work indicating how much time they used 
to work on the program# 

5. Instructions for administering the above instruments in 
field testing are as follows; 

a. Give students pre-test on material to be covered by the 
program. 

b. Hand out programs to students within a day after pre- 
test with instructions and time sheets* 

c. Sivf students designated amount of time to complete 
prq^rams. 

d. Collect completed programs and time sheets from student 

e. Give students post-test within a day or so of their 
completing the program. 

f. Give students the Student Evaluation Questionnaire 

within a day or so of their having completed the post- 
test. ^ 

g. Fill out of the Instructor Program Evaluation. You may 

wish to start to record your reactions while the stu- ! 
dents are still working on the programs. ! 

! 

h. Ship programs, pre-and post-tests, student questionnairi 
time sheets, instructors questionnaires, back to the I 
programer who will handle the data from that point. 

About processing field test data Dr. Daniel Smith made the 
following suggestions. Data from field tests are usually of such 
quantity that they tell the program writer more than he can assimi- I 
late about the program. The task is to sample the data selectively. 
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For samples of the actual instruments 
See Appendices B-1 and B-2. 
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and then to examine particular questions using a more extensive 
sample if it seems warranted. Therefore, I would suggest that the i 
program responses be divided into thiee categories: top students, | 

average students, and low students. This can be done on the basis ; 
of combined grades and SAT scores to be obtained from the Personnel 
Data Roster. Then I would suggest that three students be selectee 
at random from each level from each campus, their program responses’ 
and pre-post-test responses be examined in detail. Keep a record > 
of hypotheses concerning the program growing from this examination.! 



A count of errors, as well as pre-and post-test scores ‘and i 
gain scores, should be kept for the students of the sample at each 
level and for each campus; this will make it possible to draw some 
broad conclusions of a comparative nature across all programs (thus 
names of students should be entered along with the data). 

( 

For purposes of revision, on the other hand, such data are 
less important than the hypotheses the programer forms by consider- 
ing the nature of the errors. The fact that errors have been made : 
on a particular item vjill not necessarily imply that a revision of 
that item is necessary; it may imply instead a revision of certain j 
prior frames or sequences of frames. Indeed, the nature of errors 
will be only one of several factors considered in revising: other ; 

factors will be fhe experience of the programer in programing, the ■ 
feedback coming from editors, and feedback from si;bject matter ex- ; 
perts who reviewed the programs. Of course, there' will be some * 

cases where one will want to examine a larger number of cases in : 

order to determine more accurately the nature and extent of errone-! 
ous conclusions on a particular item or set of items. With these i 
instructions and suggestions in mind, GLCA faculty members field i 
tested programs at their several institutions.^^ Programmers j 

examined the results thus obtained, and, in light of them, many 
revised their programs. | 

Fourth Liaison Committee Meeting 

The fourth Liaison Committee Meeting was held at the Airways 
Motel in Columbus Ohio on April 2, 1965. All of the colleges ex- 
cept Wabash were represented. 

The first item for discussion was means for improving the 
production and field testing of programs for 1965. The committee | 
decided to have the grant period run to April, 1966, since most ' 
programers' work would extend more than six months beyond the i 

summer. They also decided to ask for an outline of material before! 



Mhi 



For a list of programers and field testers, see 
Appendix B-3. 
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1° programer would be ready to work to 

ctltr «"'«"» of objectives during the workshop. ALtheH^ 

he?n L ^ securing field testers and consultants to 

help programers . o«o.i.eiiita to 



°f **’? P»^o3eot's most difficult problems involved conduct-, 
jqr^ of the 6LCA -produced programs during the Fall of 

i ^^itl to^cffevo:- Of Programed®Ins?r««ion 

nrovArf ® presented to the Committee by the Director 

P to be too unwieldy, and the committee spent a long time* dis- 

DrosrAL°r®'' evaluation. . One suggestion was tLt using 

ditio^ but lectures be tried out as the only con- 

dition, but varied under a number of teaching situations. Any nro- 

fommi^ evaluate the programs would be expected to^ 

commit himself for a period of two weeks in which he would teach the 

wAQ^A^ t"^ ^®® ^^® according to the evaluation design. Ther. 

;jhat as many controls as possible be\pplied. 

L foimula5e arfverall ®®? ® research design committeel 

1 u evaluation plan. The designs drawn from the! 

mK ^®®*^ the Director to recruit 6LCA faculty I 

thP^r? evaluation. The Director would then work with 

1 -hA persons to refine a particular design, fitting both i 

An evIluatLn'^o^^f^^®" instructors' teaching situation. | 

^ conference was planned for September, 1965 to provide 
all the program evaluators brief but intensive training in the nro- 

erL^Chaptere!""' GLCA-produced programs is deLrib 



The committee also set up the following series of deadlines 

1. First programing workshop, June 14-16 

2. Sepond programing workshop, July 12-14 

«. Research workshop September 1965 for evaluation of GLCA* 
1964 programs 
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Early Spring, 1966, Liaison Committee 

Late Spring, 1966, Final Reporting Conference 
Second Round of Program Writing 

nronaHn^^ the second round of programing in 1965, an administrative 
procedure similar to that of 1964 was used without, however, having 

writina^r,!!«^^ Conference to generate proposals and enthusiasm for 
writing programs. During the Winter of 1965, the Director inter- 
viewed prospective programers on their campuses leaving application 



forms with them. A selection panel consisting of Drs. Loche Van 
Atta, Daniel Smith, Donald Beane, and the Director selected the ' 
program applications that seemed most promising. | 

i 

The criteria used in selecting proposals for programs to be 
written in the Summer, 1965, were as follows; j 

1. Does the proposal give evidence that the person has a 
clear, concise idea of the content of what he wants to 
program? 

I 

2. Does the description of the program give evidence that j 
the person has a clear, concise idea of the process of 
programing? 

3. Is the scope of the material to be programed sufficiently 
delimited? 

II 

4. Does the person give evidence of having selected material* 
which is programable and which can be hahdled by a pro- 
gram better than by some other instructional means? 

Interestingly enough only about 30 applications were received 
as opposed to sixty the year before. Twelve were selected. 

I 

The method of selection used both in 1964 and 1965 had one | 
weakness, namely, there was no panel of content specialists to 1 

evaluate the content of the application for programing. The se- i 

lection panel consisted of men who were expei'ts in programing but 
not in each subject matter area. The same general agreement be- 
tween programers and Project were in effect as in 1964. 

The concept of the team was developed more fully in the second 
round of programing in 1965. In this procedure content specialists 
early consulted with the programers. However, even at this point it 
was probably too late. Sympathetic content specialists should 
probably sit down at the time of the programer's writing of the 
proposal in order to give him help. at the very beginning. 

In response to a question asking; ”Do you have any suggestions 
f^v improving the effectiveness of content editing?" two of the 
1965 programers said the following; 

- - The most efficient way to use content editors would have 1 

been to let them look over the subject matter outline be - L-. 
foye starting to write the program at all. 1 

iiM I 

i 

- - Content editors should be required to sit down with the 

programer and go over the material step by step. ■ 



The idea of a team of scholars from a given discipline drawing 
up a proposal for a program was set up as one of the goals in the I 
contract. The goal was not reached. The kind of cooperation im- * 
plicit in such a goal requires a high level of cooperation that does^ 
not presently seem possible in 6LCA. j 

First Session Summer Workshop, 1965 . ] 

t 

The firjt session of the summer programing workshop was held | 
at Indianapolis Airport Hotels June 14 - 16. The entire session was 
devoted to the writing of behavioral objectives, criterion frames, 
and teaching frames. 12 

Four group meetings were held as follows: Monday afternoon 

Dr. Daniel Smith talked about th<? prscess of programing and concept 
formation. Later Dr. Morton Shagrin was the moderator of a brain- ‘ 
storming session in which the various problems of the programers 
were discussed, particularly the writing of behavioral objectives i 
and criterion frames. On Monday evening the film "Progra.ied In- 
struction: Developmental Process” was shown and discussed. Three ‘; 

presentations was too much for one day; the brainstorming session i 
might have bettor been left out. i 



On Tuesday a group session was led by Dr. William Stephenson \ 
on the ”eg-rul” process of frame writing. (See Chapter 7) Dr. I 
James Cook also handed out a program on "Developmental Testing." } 
On Wednesday Dr. Cook continued with the "Developmental Testing" 'i 
program. Mo more group meetings were held. 



Second Session Summer Workshop. 1965. 



The second session was held again in the Ind ianapol is A irpor t i 
Hotel on July 12-14. On Sunday, July 11, briefing session was held ) 
at which time all the editors were present along with Dr. Donald 
Smith and Mr. George Geis from the University of Michigan. 



On Monday morning the editors, including Smith and Geis, 
worked with the pragramers as soon as they came in* The “ 

editing continued throughout the day. The programers were preparing! 
to test their programs on student subjects from Butler University. 



12 



For the agenda of 
196S see Appendix 



the First Training Workshop Summer 
B-4. 
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^Throughout the morning on Tuesday there was continued editing 
activity with the four 6LCA editors* By one o'clock the students 
from Butler University were on hand and were assigned to the pro- 
gramerSf each programer working with one student for a period of an 
hour and a half. i 

Programers varied in their openness to the responses the stu-; 
dents made to their programs. One programer, for example, kept • 

adding verbal instructions to his program and learned practically \ 
nothing from the student's response to what his program conveyed. 
Two others were very sensitive and perceptive to places where the 
students seemed to bog down. . i 

f 

The GLCA editors held a discussion with programers between i 
their sessions with the test subjects. The editors pointed out to I 
the programers the deficiencies in their approach. It was noted, i 
for example, that when a student makes a mistake and a programer 
says, "You were wrong there, weren't you?" it is likely to inhibit i 
the student from giving further information. If, however, the pro-’ 
gramer had said at that point, "The program was not very clear at ■ 
that point, was it?" he would elicit much more information from i 
the student about the program. • 

On Tuesday evening Drs. James Cook and Morton Schagrin led a | 
discussion on questions raised in the developmental testing. j 

! 

On Wednesday the group met, and the programers were given 
4x6 cards on which they were tentatively to describe their pro- j 
grams. The group then discussed handling the content editors and 
the program. The programers were asked what questions they wanted S 
the content editors to answer. The programers suggested a number ? 

of questions which the director put into a memo form and sent out i 

to them a couple of weeks after the workshop. ; 

The group then discussed what programing could teach them 1 

about teaching in general. Their main observation was that the 
setting up of objectives was an extremely important and valuable 
lesson for teaching in general. They did not, however, discuss the 
importance of empirically testing what they wanted to teach. 

Neither did they talk about the importance of analyzing the subject' 
matter in terms of the task that the student has to perform on it. . 
The rest of Wednesday was spent in writing frames, reviewing what * 
they had done with students, and editing. The conference closed 
at 5 o'clock. : 



Evaluation of Summer Workshops, 1985 

The twelve programers who participated in the Summer Workshops 
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in June and July of 1965 evaluated the two three day workshops on a 

summIrnerhelo«®^ received at the workshops. Their evaluations' are 
e below, with excerpts from several of them. The same 
questionnaire was given each time. ! 



that it^Kaa ^ general evaluation of the workshop, everyone stated 
tho fwisi valuable, a good experience, or useful. Eight of 

rece experienced assistance they ’ 
durtir editors, who themselves had written proar^rs i 

app^oacLs? ® strengths and ; 



- - The editors guided without dominating. 



and^evening^^ * * * "iftstant editors” available day 



" ■ editor was both critical and encouraging. 



the mentioned the informativ© general sessions 

the helpfulness of presentation as well as editing of Dr. Daniel 

wruL nro^ra:: gh time for inuliZi w^^rL 

Qhon ^ px^ogramers felt that a three day work 

wi?h persons felt that work was broken up 

with general presentations. ^ 






Presentation of constructing terminal frames was helpful, 
- - Egrul and RuXeg concepts • have been helpful. 



®®s®i®ns* by and large, were good. Some of them 
frame writinr^ P**ogress I thought I was making in 



Basic sequence of topics was good. 



were similar h«t ^*^*>18 same question after the second workshop 
anrcrltici^m but were uniformly more certain both in commendatiL 

editors on h!nd most programers mentioned the value of having 

„ Half of them mentioned the value of the Tuesdav 

afternoon session with test subjects and the following discussion 

thrunivarsitrof Michisar 0<>«ai<i Smith from •; 

fnL thl Michigan. Three persons indicated receiving help i 

®*"**’®^ aesBions, one mentioning he would have liked®more^ l 
wLkinriim!" P’'"®*’®™®’'®* however, would have liked more individual 

! 



32 








- - Being able to consult with four editors was invaluable, i 

as was the opportunity to test. | 

i . 

- - I liked being able to take a question to a number of i 

people • 

- - Useful, particularly the testing session. ; 

' ! 

- - Proportion of time spent on talks and individual work was, 

. better than first session. | 

A second question asked: What new insights or understandings 

of programing occurred to you during and since the workshop? What 
slowed you down or held up progress? 

Answers to this question after the first workshop were varied. 
Two persons mentioned skills gained in writing a program rather 
than^a textbook. Several mentioned insight into making clear ob- 
jectives, and another person added the ability to put objectives 
into operational terms. Five of the programers referred to skill 
gained in understanding how a student learns as over against how a 
teacher teaches. Half of the programers mentioned insight into 
some programing jiechnique: writing criterion frames, terminal 

frames, or discrimination. 

- - I realized more clearly the relevance of making an explich 

it statement of objectives and difficulty of making a 1 

workable one. | 

- - I soon learned to stop trying to write a textbook and 

begin a program. 

j 

! 

- - I saw relationship of techniques to problems. I 

- - I was held back on a content problem, i.e., had chosen j 

too large an objective. Breakthrough came when I dis- I 

covered an interim method by which it could be solved. 1 

- - Helpful to begin with construction of terminal frames. ' 






- - Objectives set by ardent exponents of programing are too ! 

ambitious. i 

i 

- - I've grown to appreciate the plight of the student. I 

. ; i 

” " Criterxon after frames as a method have helped a lot. I 

After the second workshop this question was ansxfered more in | 
terms Of progress on particular programs than in general insights | 



I 




f:\ 

i'; 



> J . 



I 



li 









V ' 



Pi 



I 






fry 









I ^ 

ERIC 









\ i 



into programing or in teaching and learning* 

“ - After the second workshop I rewrote the entire program* 



- - I observed how differently the two girls I tested set out 
their workf and the difficulty of accommodating a broad 
range of ability* 



• - Most valuable suggestions were those concerning use of 
discrimination and those on format* 



- Disappointed with editorial help* Some of it seemed con- 
tradictory or at best tangential* This was likely, how- 
ever, because of the specialized nature of my project. 



- - Discrimination frames was greatest revelation to me* 
Below i s a list of programs produced in the summer. 1965 



A PROGRAM IN COMPOSITION by Fred Bergman 

Departmeii'^ c^f English, DePauw University 



MAKING SENSE OP EXPERIENCE by Owen Duston 
i Department of English, Wabash College 



FREEDOM VS* FREEDOM by Robert B* Fichter 

Department of Religion, Ohio Wesleyan University 



j PASO A PASO by Renato J* Gonzales 

Department of Spanish, Albion College 



A 



ALTERNATIVE L06IC.S by L* H* Hackstaff 

Department of Philosophy, Wabash College 




LABORATORY SUPPLEMENT FOR CHEMISTRY 11 and 12 by Alfred Henderson an 
Laurence Strong, Department of Chemistry, Earlham College 



MUSICAL RHYTHM by Leonard C * Holvik 

Department of Music, Earlham College 



POLITICAL DIST^SITION OF ADMINISTRATION by William L* Morrow 
Department of Political Science, DePauw University 



AN INTRODUCTION TO MODERN PHYSICS by B* R, Russell 
Department of Physics, College of Wooster 



A PROGRAMED REFERENCE GRAMMAR OF ELEMENTARY FOR SPOKEN AND WRITTEN 

by Richard Strawn, Department of Romance Languages, FRENCH 

Wabash College 
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ELECTROPHYSIOLOGY OF NERVOUS TISSUE by Francis W. Yow 
Department of Biology, Kenyon College 

CRYSTALS; AN INTRODUCTION - A PROGRAMED LABORATORY AND INDEPENDENT 
by John White, Department of Earth Sciences, STUDY UNIT i 

Antioch Collage 

Field Testing 1965 Programs 

The extension of the contract called for a conference . to be 
held in September, 1965, for the programers and the persons who 
were to field test their programs. However, because every pro- 
gramer was at a different stage in the development of his program ; 
at the end of summer and because there was such a wide variety of | 
programs being produced some of which used other media besides j 

straight verbal material, it was not considered desirable to hold 
one field testing conference. There was very little that the pro- 
gramers had in common that would warrant such a meeting. Instead I 
each programer was authorized to meet with two field testers and | 
to review with them the procedures for field testing at a time and ! 
place convenient to both programers and field testers. | 

i 

The Director supplied the programers with sufficient numbers i 
of Student Evaluation Questionnaires and forms for collecting data ! 
on the students wbo took the program in the field test as well as ■ 
field testing directions. The programers were given the responsi- i 
bility of distributing these materials to their field testers I 

themselves. = ' 

I 

i 

Evaluation of 1964 GLCA produced programs j 

The extension of the contract called for the evaluation and ! 
comparison of th© use of programed instructional materials prepared! 
in 1964 in as great* a variety of instructional situations as possi-| 
ble. In the fourth Liaison Committee meeting held on April 2, 1965’ 
the decision had been made to evaluate the 1964 GLCA produced pro- ; 
grams under conditions that were as controlled as possible. 
According to that decision a small number of programs would be 
evaluated under a limited number of well controlled conditions j 

rather than having a large number of programs tested under a large ! 
number of poorly controlled conditions. ' 

j 

The process of selecting programs to be evaxuated was as | 
follows; A selection panel consisting of Professor Celeste Me I 

Collough from Oberlin College and Dr. Daniel Smith from Earlham 
College along with the director inspected all of the programs that I 
were produced in 1964 and made a selection of five programs that i 

they considered to be the best suited for evaluation. The major 
criterion of selection was the excellence of the program. 




In addition, however, an attempt was made to represent all of the 
major areas of a liberal arts curriculum- the natural sciences, the 
social sciences, and the humanities. 

The following programs were selected: 

1. POETRY: METHOD AND MEANING, by James W. Cook, Department 

of English, Albion College 

2. LANGUAGE OF LOGIC, by Morton Schagrin, Department of History 
of Sciences, Denison University 

3. BIOCHEMISTRY FOR BIOLOGISTS, by William K. Stephenson, De- 
partment of Biology, Earlham College 

4. STUDIES IN THE GOSPELS, by Robert Montgomery, Department of 
Religion, Ohio Wesleyan University 

5. AN INTRODUCTION TO SYSTEMATIC ANALYSIS IN POLITICAL SCIENCE, 
by Lois M, Pelekoudas, Department of Political Science, 
Antioch College. 

' 

j 

The programs are described in detail and safinple pages display-1 
ed in Chapter 3. . 

In the spring of 1965 the Director visited eaph of the twelve 
GLCA colleges to display all of the GLCA programs that were pro- 
duced in the summer of 1964. The purpose was to recruit faculty 
members to evaluate the selected GLCA programs. The Director pre- 
pared a memorandum to prospective evaluators of GLCA. programs in 
Departments of English, Mathematics, Natural Science, Philosophy, 
Psychology, Biology, Religion and Economics, and Political Science 
(those departments into which the selected programs fit). The 
memorandum described the general basic question which the evaluation 
was designed to answer: How can programed materials be most effec- r 

tively used in college teaching? Four specific questions were for- 
mulated to elaborate the basic question and to serve as the focus ! 

of the._ej5^aljiciti^^ the programs. They were described in the | 

memorandum lis" follows : j 

1. Do programs teach as effectively as textbooks or lectures | 

covering the same material? | 

i 

2. What classroom activities such as discussion or lectures 
following the students* use of the program most effectively 
capitalizes on what the students have learned from the program? 

3. Is the program more effective as an instrument for the ! 

acquisition of knowledge or information or as a device for reviewing! 
information acquired earlier by some other methods? 
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4. What effect does intrinsic or extrinsic motivation have onj 
learning by programed materials? t 



One additional question was asked, namely what characteristics 
of the students affect their learning via programed materials. j 



The memorandum also described requirements for prospective 
evaluators. Anyone desiring to evaluate one of the GLCA programs 
was expected to meet the following conditions: 






1, Has access to 50-60 students during the first semester or 
term of 1965 in a course where the program to be evaluated can be • 
used appropriately and where the students can be separated into i 

relatively independent groups. For testing three or more conditions 
three or more groups will be needed. < 



f 



2. Can give approximately 10 days to the evaluations and re- 
lated activities during the first semester. 



3. Will take responsibility foi* the collection of background 
data of students particpating in the experiment. 



4. Will attend a two day research conference toward the end 
of the summer of 1965 at which time final arrangements for con- 
ducting the evaluations will be made. 



5. Will attend a one 
1966 at which time results 
ported out and discussed. 



day conference in 
of all evaluation 



winter or spring of 
projects will be re- 



6. Will select in consultation with the coordinator and ad- 
here to one of the evalu tion designs. 



Four evaluation designs were then described in the memorandum 
Each of the evaluation designs approximated normal classroom situa- 
tions and yet were set up to evaluate questions above which were 
pertinent to the teaching interest of GLCA faculty members and to 
their concerns about programed instruction. Chapter 8 contains a 
full and complete description of the evaluation designs. 



For his services in conducting the evaluation the faculty - 
rmember was offered a stipend of $500 plus funds for materials and 
clerical help in collecting data on students. It was anticipated 
;that about ten days of the instructor's time would be required for 
the evaluation and all that pertains to it including conference 
'time. It was also promised that the data would be processed by 
•computer. The Computer Center at Kalamazoo College was used for 
this work. 
















Beane, on leave of 
as Evaluation Pro- 
evaluators except 
He reviewed* with 



i-h» foifras for participating in evaluation of one of 

instruction- "^^6 faculty members and 

xn^rrueexon^ were given to have them turned in by May 7. 196S 

colLLr rrT applicants whe^ he visited tS^ 

non-GLcrco?T!a! instructors in eight QLOA colleges and one 

GLCA college agreed to evaluate the selected GLCA programs. 

During the Summer of 1965 Professor Donald 
absence from the College of Wooster was selected 
ject Coordinator. He conferred with each of the 
wo w ©were off campus until early in September. ue reviewed wi 

foLow^in^o^a^ procedures they would be expected to 

a nnv evaluate the program. Each evaluator was given 

of the spelling out in detail for each day 

Fnr.m r "hat the instructor was expected to do. A sample 

a^ifreHnf th« h”: categories, the ques^orfo i 

evaluatL !n ^he evaluation design differed from one ■' 

evaluator to the next. In addition to receiving a oonv of the I 

from“thrCoordinLord -i!® a^so receiv^^a memorandum ! 

he Coordinator describing the general responsibilities of all I 

program evaluators. The memorandum describing the general 

L"z:rof'i as"" "“r-" “ the 

individn^^!eL®®°^ evaluator had participated in a minimum of two I 

Director in the early spring or 
threvalufi!L°h!a"“- '"a® Coordinator later in the summer. All of i 

thei^ i u*:tL ir^::s:^es :vfiL^^:ri:?ain%d^^^ 

^ ^hrdor%i. , 

f 

project!^^ following instruments were developed for the evaluation 

Stude nt Data Roster (See Appendix E-6) 

^^^^^^^^^^."Method Eval uation Questionnaire (for students) 

(See Appendix E-6) 

I 

3. Instructor Program E valuation Form (See Appendix B-1) | 

4. Each programer produced an equivalent set of ore and nnai- ! 

test questions. See Appendices E-1 to E-5. ^ 






I 



.. . le aching Method Evaluation Questionnaire, was developed out! 

and'’!?h 1954 field testing of GLCA programs ^ 

Thp of commercial programs during the fall of 1964 I 

P® program Evaluation Form is identical with the one 

developed for the evaluation of commercial programs in 1964. i 

t 
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was held a Program Evaluation Training Conferenc^ 

n« 4 -K Holiday Inn of Columbus Airport, Columbus, Ohio. 

on ^he GI^CA DeHaan reported on the progress 

on the GLCA Programed Instruction Project. Dr. Donald Beane 

described the procedures for the fall evaluation of 1964 programs. 

the first day the program evaluators Lt^with 
discuss the programs with them. Late in the 
the programs, tests, and questionnaires and other evalu- i 
ation materials were distributed to the program evaluators. On the! 

thP^locr’ir^"^”® conference Professor Morton Schagrin discusseid 

JeLen^of and evaluation of programs and Mr. William , 

I ^°'"P^ter Cente^ of Kalamazoo talked about the com- I 
p er treatment of date and its interpretation. The agenda of the I 
program evaluation training conference is found in Appendix B-7. 



In Appendix B-8 
ators conference. 



is a list of 



persons who attended the evalU' 



As a result of the discussions at the 



= ' program evaluation i 

conference Dr. Donald Beane, coordinator of the evaluations 
project sent out a memo clarifying some points that required I 

5c all the evaluators were to observe. His memorandum { 

is given in appendix B-9. | 

j 

n€ i-h September 24, 1965 a conference on statistical treatment ! 
th^ ^ata that was held in Columbus, OlUo. Attending 

the conference were Drs. Robert DeHaan, Donald Beane, William 
cansen, and Sam Cho. The latter is a member of the Psychology De- 

statistical questions in the handling of data. 

^ Incidently it is worth noting the difficulty involved in 
rying to interest statistical consultants in problems of the GLCA 
from^5"d-" "-"<^tor called on stListics s^J^lis^s 

Pi^tchu^aK^^ State University, University of 

cn ’-^55 Oberlin College. The consultants either claiLd to 

couiri were over committed and 

5 as a result of the conference a 

detailed procedure for running an analysis of variance for each of 
the designs and the variables to be involved were agreed upon. 



The rest of the first semester and the early part of the i 

second semester was devoted to administering the evaluation designs 
his was coordinated by Dr. Donald Boane. Results of the evalua- > 
tion are reported in Chapter 6. | 

♦ ? 

1 -K^ report stress has been laid on the actions taken by f 

the Director and the Coordinator of the Evaluation Project to be^as 

evaluators knew what they were Opposed * 
o do before the evaluation got underway and then adhered rigidly 
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to their instructions once the evaluation was started. If anything 
was learned from the evaluation of the commercial programs in 1964 
it was that poorly conceived evaluation designs and incomplete data 
produced inconclusive and unsatisfactory results. In summary, the 
steps that were taken to assure interpretable and comparable results 
are as follows: 



1. Use of the same program in four different evaluation 
designs . 



2. Use of carefully prepared pre and post tests. 



3. Two individual conferences between each evaluator and the 
program director or the coordinator of the evaluation project with 
some unavoidable exceptions. 



4. A group conference on evaluation. 



5. Agreement reached between the director and the coordinator 
a detailed evaluation design including such things as how to 
the pre and post tests, what instructions to give, how to in- 
troduce the progi^am to the student. 



about 

score 



6 . 



Agreement reached on what data is to be collected and in 
what format it is to be recorded. 



7. The use of common instruments to collect the instructor's | 
evaluations the program and the students reaction to it. * 



Fifth Liaison Committee Meeting 



The fifth Liaison Committee Meeting was held in the Holiday 
Inn, Columbus Airport on April 1, 1966 from noon until 5 P.M. All 
the colleges were represented except Oberlin and Wabash. 



The committee received a preliminary report of the evaluation 
of 1964 GLCA programs from Dr. Donald Beane. The report included 
the results of the analysis of variance and a brief analysis of four 
of^the most important questions on the student evaluation question- 
naire. The committee responded by suggesting certain changes and 
additions to the report. The report will next be presented at the I 
final evaluation conference on April 29. 



The Director made a verbal report of the study of the impact ^ 
of programing ljhat he and Dr. Clarence Leuba are conducting. There j 
were no results or conclusions, however, to be reported at this time 



f 



The main item of business for the Liaison Committee was a 
review of the agenda of the Final Reporting Conference planned for 
April 29 and 30 in the Holiday Inn of Columbus Airport. The Liaison 
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persons reported considerable difficulty in recruiting participants 
for the conference. One of the major difficulties is that the 
conference runs for a day and a half. For many of the college 
people this would involve being gone from home for two nightSf the 
Thursday night proceeding the conference as well as the Friday 
night. Very few wanted to give up that much time for the confer- 
ence. The discussion resulted in a recommendation that the con- 
ference be reduced from a one and a half day conference to a one 
day conference. The committee agreed that it would be better to 
pack one day full than to have a more loosely structured conference 
extending for a day and a half. After considerable discussion it I 
was found that the agenda for the conference could be worked «out on 
a one day basis. 



The Liaison Committee also strongly recommended that each \ 
college send at least one member of the newly formed 6LCA faculty ! 

council to participate in the final reporting conference. Since \ 

the Faculty Council will very likely be the body that recommends • 
next steps to be taken on the project it was felt that as many * 

members of the council as possible be on hand to hear the reports j 

of how the project has been conducted for the past two and one half! 
years. This recommendation was accepted readily by all the membersl 
of the Liaison Committee. 

It was recommended that a small task force consisting of one | 
representative from each college continue on through the Friday { 
night of the fin^l reporting conference and if necessary continue | 
working until Saturday noon to formulate recommendations for the j 
extension of the project. These recommendations i?ould be addressed 
to the GLCA Board of Directors and to the Faculty Council. The 
task force would carry on the work of the conference which had 
originally been placed in the hands of the total conference. This 
suggestion of having a task force carry on was the major solution j 
to making the conference a one day conference rather than one that 
went on for a day and a half. 

Each Liaison person was given 10 copies of a preliminary draft 
of the Final Report of the Project to distribute among potential | 
participants on his campus. In addition preliminary draft copies j 
were sent to potential visitors from other small liberal arts I 

colleges. >. 
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Final Reporting Conference 



f 



The Final Reporting Conference was convened on Friday, April 
29, 1966 at the Holiday Inn, Columbus Airport, Ohio. The purposes 
of the Final Reporting Conference were two fold: To report out the 

results of the Programed Instruction Project and to develop plans i 
for furthex' study and improvement of instruction in the colleges j 
comprising the Great Lakes Colleges Association. The f’.ial reporting 
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conference was one of the major instruments for desseminat ing the 
the results of almost three years of work on the project. 

Seventy persons attended the conference, fifty five of the par- 
ticipants came from eleven of the GLCA colleges. Ten visitors also i 
attended the conference representing the Center for the Study of j 
Higher Education, University of Michiganj Commission on Institutional 
Cooperation; Principia College, Elsah, Illinois; Center for Research! 
on Learning and Teaching, University of Michigan; Associated Colleges! 
of the Midwest; Lily Foundation; Calvin College; the University of / 
Cincinnati. The conference staff and speakers consisted of Robert 
DeHaan, Director; Mrs. Robert DeHaan, Conference Secretary; Profes 
Clarence Leuba, Wright College Campus; Professor Donald Beane, 

College of. Wooster; Dr. Eldon Jolmson, President of GLCA. The list i 
of conference participants and visitors found in Appendix B-10. | 

I 

The conference displayed approximately two dozen programs that \ 
were produced in 1964 and 196a under the auspices of the Programed ! 
Instruction Project. The display attracted a great deal of attention 
during the conference, partly because of the attractiveness of the i 
display itself. Small cards identified the programs by title and j 
s'^tbor. Programs that had some visual aspects to them such as the j 
art program or some auditory dimensions such as the music programs 
attracted the greatest amount of attention. 

I 

The agenda of the conference was divided roughly into two parts* 
The morning sessions were given over to reporting the results of the '■ 
project; the afternoon was given to planning for further steps to be ! 
taken. (See Appendix B-11 for the complete conference agenda). The | 
project Director, Robert DeHaan opened the conference by giving a } 
brief outline of the purposes of the project and what it had < 

accomplished to date. He covered s ome of the highlights that are J 
found in Cnapters 1, 2, and 3 of this report. He then introduced the‘ 
first speaker of the morning. Dr. Donald G. Beane of the College of ' 
Wooster Coordinator of the evaluation studies of GLCA produced pro- • 
grams. His address was entitled Evaluation of GLCA Programs and j 
covered essentially the same material that is given in Chapter 6 of i 
this report. 

! 

The second report of the conference was made by Dr. Clarence 
Leoba on the topic Applications of the Principles Underlyin g Pro — ^ 

gramed Instruction . His material is esssntially what is found in 
Chapter 9 of this report. ; 

i 

The third presentation of the morning consisted of A Summary ofi 
Preliminary Recommendations by the Director. The material presented j 
can be found in Chapters 3 and 10. The recommendations covered such ’ 
things as further preparation of programed instructional materials, ' 
the use of preparing programed materials for inservice training of ! 
teachers continued research on teaching and learning the use of ! 

• jt 
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computers in the teaching-learning 
centralized agency in GLCA for the 
teaching. 



process, development of a 
improvement of learning and 



Alter luncheon Dr. Eldon Johnson, President of the Great 
Lakes Colleges Association talked about the significance of the 
Project for the future. He reviewed the amount of cooperative 
. effort that has g@ne into the Project and the high degree of in- 
volvement of those who had participated in it directly. He also 



showed, however, that the benefits of the project were not easy to 
disseminate. Those most highly involved received the greatest 



disseminate. Those most highly involved received the greats 

benefit; there was less carry-over of the effect to others than 
had been hoped for. 



How to consolidate and spread the gains made on the Project ; 
IS the first and one of the most important problems for our I 

immediate consideration; secondly, we need to turn our attention 
to the problem of organizing ourselves for a continued attack on 
the problem of improving instruction; thirdly, the conference needs 
to recommend immediate action in the form of recommendations on ' 
how to proceed to solve the two problems above. j 



The r 
cussion. I 
into nine d 
domly selec 
different c 
groups was 
morning and 
of the proj 
to serve as 
follows : 



est of the afternoon was devoted to two rounds of dis- ' 
n the first round the total conference group broke up > 
iscussion groups in which the participants were ran- 
ted so that each discussion group contained members from 
ollegas. The purpose of the cross section discussion ^ 

to review the reports that had been made during the | 

to suggest as many ideas as possible for the extension ' 
ect . The discussion leaders were given five questions | 
guides to the discussion. The questions were as ; 



1 

2 



How can we communicate t)ie results of the Project? 



How can we interest colleagues in studying and improving 
teaching? 



3. What guidelines or principles can be suggested for future I 
developments in improving teaching and preparing teaching i 
materials? i 



4. What specific recommendations or approaches hold the 
greatest promise? 



What are other comments or suggestions for the future? 



After the discussion groups had met for some time the total 
conference reconvened and reports were quickly made from each dis- 
cussion group. The total conference then broke up into the second 
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round of discussion. This time the discussion groups were organized 
by colleges. The purpose of the second round of discussion was to 
take the suggestions and ideas that had been brought forth in the 
first round} sort out the ones that would be the most useful on each 
campus and rank them in order of importance for each campus, 
the discussion groups had spent an hour or s© discussing the 
they returned to the conference room where the recorder from 
college reported on the ideas that had been discussed in the 



After 

ideas 

each 

group. 



The first set of ideas had to do with how to communicate the 
results of the Project on each local campus. One set of suggestions, 
had to do with dissemination by means of brochures, professional i 
journal articles, alumni magazines and pamphlets. The second set of| 
ideas had to do with various kinds of meetings such as faculty meet-! 
mgs, AAUP meetings, demonstrations and exhibits. 



.Another large set of ideas centered around future developments! 
for improving teaching and learning on the local campuses. The i 

.. . j 



major ideas that appealed to the discussants were the development of! 
a central ag€incy or administration for the total Association and 
some experimentation with assisted computer instruction. 



The conference adjourned at 5:30 and the remainder of the 
evening was given over to the work of the task force. The task force 
consisted of two representatives from each college. Their names arej 
given in Appendix B-10. The task force assumed responsibility for ( 
i.'.ummarizing the conference and indicating the direction in which 
new steps should be taken. It addressed itself to two groups, the 
GLCA Board of Directors and the GLCA Faculty Council. 



The discussion centered around a number of topics that had been 
discussed during the course of the day. The possibility of setting j 
up a teaching--learning center, consolidation of the work of the 
Project, developing facilities for computer assisted instruction. ! 
As the discussion progressed the group focused more and more upon i 
the idea of a Center that would have as its purpose the involvement • 
of the college in development .and research on teaching. The dis- 
cussion v^as brought sharply to a. head with the suggestion that the 
Center be called a Center fo r Educational Development. The title 
captured the imagination of the task force. It contained a notion 
that the emphasis would be upon the development rather than upon 
research, that teaching would receive greater emphasis than learning 



I 



The purposes in definition of the Center were outlined under 
four general headings; teaching, learning, media and dissemination. 
A list of activities were classified under each of these headings. 
The task force then reviewed the list of activities and underscored 
items that could be started up immediately or established as pilot 
projects . 
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The Director was instructed t 
the task force and to present to th 
Lakes Colleges Association a propos 
Educational Development at their me 
of Directors would be requested to 
force for information and to reques 
of the Board be empowered to approv 
it is improved by the Faculty Counc 
1966. The Faculty Council would be 
the new Center in principle and to 
up a proposal on how such a Center 
The Faculty Council proposal would 
of Directors. 



o summarize the discussion of j 
e Board of Directors of the Greajt 
al for setting up a Center for | 
eting on May 9, 1966. The Boardi 
accept the report of the task 
t that the Executive Committee 
e a more complete proposal when 
il which was to meet May 18-21, 
asked to approve the concept ofi 
establish a committee to* draw ! 
could be financed and opei^ated. | 
then be presented to the Board | 



Recommendations Growing out of the Final Reporting Conference 



Pro j e 

Lakes 

port 

Colie 

A spe 

ceive 

Commi 

Counc 



The Director prepared a report on the 
ct which was presented to the Board of 
Colleges Association at its meeting on 
contained a general recommendation that 
ges Association establish a Center for 
cific recommendation was made that the 
the report for information and empower 
ttee to approve the establishment of a 
il recommends it on May 18, 1966. 



Programed Instruction | 
Directors of the Great j 
May 9} 1966. The re- | 
the Great Lakes i 

Educational Developmentj. 
Board of Directors re- j 
the Executive j 

Center if the Faculty I 



The Director presented a working paper 



on the 

truction Project at the Faculty Council meeting on 

to the 



The following recommendations were presented 
were adopted. 



Programed In- 
May 20, 1966. 
council and 



1. That the Great Lakes Colleges Association establish a C 
ter for Educational Development. 



en-j 



2. That the Faculty Council approve the appointment of a 
committee to study all pertinent phases of the establishment and 
operation of a Center.. 



3. That the committee prepare for submission to the Execut 
Committee of the Board of Directors of GLCA a proposal for the 
funding and operation of a Center. 



ive: 
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PRECEDING PAGE MISSING 

Chapter 3 

DESCRIPTION OF PROGRAMS PRODUCED ON THE PROJECT 
AND DEANS' EVALUATION OF THE PROJECT 



A total of 34 programs were produced on the Project representj- 
ing almost every department in a liberal arts college curriculum. ! 
In addition, a number of smaller programs dealing with library ! 

procedures was produced. Three other smaller programs were- also 
developed two of which utilized 8mm film loops or audio tapes, and 
the third of which used already existing German readers. Combining! 
the library programs into one program, and considering the latter | 
threo as one, it can be said that the Project produced 36 programs. | 



Detailed Description of Each Program • 

! 

1 ^ 

Below is a brief description of each program. 1 

TECHNICAL FILMS ON BASIC PRINTMAKING TECHNIQUES I 

Paul Arnold, Department of Art ' | 

Oberlin College^ Oberlin, Ohio ! 

General Objectives ; To break down the basic techniques of | 

intaglio printmaking into component steps, 
each of which is covered clearly and in 
detail in a loop film which can be pro- 
jected easily by the student when he needs 
the information. The films eliminate the 
necessity for repeated individual explan-- 
ations and demonstrations. 



Courses for Which Intended ; Problems in Printmaking 

Topics Covered ; All steps from preparation of plate through | 

printing of impression i 

Approximate Time Fsequired ; Film loops used as needed ] 

f 

— Vifhen and Where Field Tested : Oberlin College, second semester 

64-65 by beginning students whO; 
entered class ! 

____ * ^ 

It was not possible to obtain descriptions from all the \ 
programers because some of them were out of the country, some had \ 
left GLCA colleges, or for other reasons were 

1964 group the following programers did not send in descriptions of 
the programs: Preston Adams, Ansell Gooding, Peter Hawkins, Richard 

Knudten, Fred Silander, Marian Mulendore. j 


















I 



\ 

X Results of Field Test ; Film loops accomplished the desired 

results. Only occasionally was it 
necessary to give students additional 
instruction. 

GERMAN VOCABULARY THROUGH COGNATES 
Robert Brewster, Department of German 
Earlham College, Richmond, Indiana 

General Objectives ; a) to provide the beginning student of 

German with an active vocabulary of 200 
words, over-learned visually and aurally 
and 

b) to have the student learn inductively 
ten of the main consonantal relation- 
ships between English and German 
through these 200 word examples, and 

c) to help the student recognize new Ger- * 

man words on the basis of these con- | 

sonantal laws. I 

Courses for which Intended ; Beginning German, after the first ^ 

three or four contact hours of * 

pronunciation practice of German' i 

sounds; a pre-reading program, , 

before textbook contact hours withj 
German. 1 

Topics Covered ; a) 200 German words related to English 

equivalents by 10 consonantal laws. 

» 

b) 140 Additional German words based on these ; 
laws. I 

j 

Approximate Time Required ; Six or seven hours for college \ 

students. i 

I 

I 

Prerequisite Knowledge ; None j 

When and Where Field Tested ; Hope College, Holland, Michigan \ 

by Dr. Gearhart, Department of i 

German, September to October, ] 

1964. 

Results of Field Test : Material too easy for college level. 
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POETRY, METHOD AND MEANING: A PROGRAM IN POETIC ANALYSIS AND } 

James W. Cook, Department cf English CRITICISM: 

Albion College, .Albion, Michigan : 

! 

I 

G e neral Ob]]ectives : 1. Student should use critical vocabulary} 

when writing, thinking, or talking. 
about poetry. 

2. Student should identify figures of 
speech and tnought when they occur in 
a poenn. 



3. Student should be able to analyze and 
specify the contribution of figures of 
speech and thought to a poem's meaning, 

4, Student should be able to posit multi- 
level interpretations of a poem. ; 

Courses for which Intended : Introduction to Literature or any* 

course in which poetry, especially 
lyric poetry, is to be a primary ^ 
concern. : 



Topics Covered : Basic poetic figures of speech and thought 

and the concepts of metaphor, symbol, and , 
image as they interact and contribute to the ! 
meaning of a total poem. i 

Approximate Time Required : Four to eight hours 

Prerequisite Knowledge : Freshman or sophomore college stand- ; 

ing or advanced high school student : 
without previous experience. 

When and Where Field Tested : Ohio Wesleyan, Fall '64 Robert j 

Ross Kalamazoo, Spring *65 Wal- ■ 
ter Waring Albion, Fall *64 J. 

Wo Cook ■ 

Results of Field Test : Used to revise program. 

ANALYTIC GEOMETRY: THE LINE 

Thomas A. Davis, Department of Mathematics 

DePauw University, Greencastle, Indiana ! 



.. General Obiectives : This program is designed to be used in a 

college level course on analytic geometry 
and calculus along with a text in 
analytic geometry and calculus or a text 
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in calculus. Students taking these coursers 
have a wide range or previous training. ! 

This program will allow each student to ^ 

spend as much time on each topic as he J 
needs, to learn the material he does not f 
know. ! 



Courses for which Intended : Analytic Geometry 

and 161 Beginning 
college freshmen 



and Calculus 
course for 



15l! 



Topics Covered : The first unit of the program 

material usually found' in the 



contains the 
first 50 or 60 
Analytic Geome- 



; 



r- 






pages of college textbooks on 

try and Calculus: coordinates, inequalities, , 
absolute values, absolute values and inequal- ’ 
ities, directed distance, distance formula, i 
midpoint formula, slope of a line, parallel ! 
and perpendicular lines, the angle between two' 
lines, graphs and equations, intercepts, * 

symmetry and asymptotes, and the straight line:. 

i 






,r 



Approximate Time Required : Eight to twenty hours--mean twelve 

hours. 



Prerequisite Knowled ge: High School Algebra 

and Where Field Tested: DePauw--Fall ' 64 



Winter *65, 

Fall '65 

Denison--Fall ' 64--Sterrett 
Albion--Fall '64--Moore 
Earlham--Fall *64--Hahes 



Results of Field Test : Used to revise program. 



CONSTRUCTING THE UNIT CARD 

FILING IN THE LIBRARY PUBLIC CATALOG AND SHELF LIST 
CHOICE OF MAIN AND ADDED ENTRIES 
CHOICE OF SUBJECT HEADINGS 
CLASSIFICATION 
;J. McRee Elrod, Library 

.Ohio Wesleyan University, Delaware, Ohio 



General Objectives 



§m 



To Prepare library d'erical and subpro- 
fessional personnel to perform a higher 
leve of technical processing functions as 
listed in the titles of the programs; to 
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allow the library school teacher to j 

teach these clerical and subprofessional i 
functions outside the regular classrooin. 

jf-Q^^s es for which Intended ? The technical processing area of 

the library's inservice training 
program; the library schools’ 
technical processing courses. 

ARpyoximate Time Required ; Eight to sixteen hours j 



Knowledge ; Typing; senior high reading level of ! 

English; general library orientation ■ 
such as given college freshmen. 

AqP ^ pA Where Field Tested ; Clerical workers of the Ohio 

Wesleyan U, Library Catalog De- ' 
partment and The Defiance Coll- ; 
ege Library; 1964-65. j 

Results of Field Test ? Revisions of programs for clarity. 1 



CAPITAL BUDGETING, METHODS OF RANKING PROJECT PRO'POSED FOR INVEST- i 

„ „ ■' MENT I 

Vant W. Kebker, Department of Economics j 

Ohio Wesleyap University, Delaware, Ohio | 

i 

G eneral Objectives : To help students understand why the | 

method of discounting expected cash flows| 
of income is better than other methods i 
evaluating and ranking proposals that in-i 
volve investment of capital.. 




pQ^^ses f or which Intended ? Financial Management, Business 

Finance, Corporation Finance 

_ Topics Covered ; Concept of discounting payments to be re- 
ceived in the future, methods of measuring 
money available to recover invested capital, 
methods of measuring results obtained. 



i 

( 



Approximate Time Required ; Eight hours 

Prerequisite Knowledge ; Accounting 

and Where Field Tested .* DePauw University, Fall '64-- 

Selander Ohio Wesleyan Universit 
Fall '64--Kebker, Fall *65, 
Winter '66 




j 



Results of Field Test? Revised Program 



‘LIBRARY OF CONGRESS CARD ORDER ROUTINE 

Feter Kidder, Library j 

I Kenyon College, 'Carabier, Oh-io } 

! 

General Objectifies : The program is designed to prepare people ' 

to do • preliminary bibliographic searching 
in connection with orcering LC cards; order 
th em ; check them in, and do necessary 
follow-up work. i 

! 

Courses for which Intended ; For use in training clerical* ! 

personnel in the acquis.it ions and | 
cataloging departments o€ a library*. 

Topics Covered : General introduction to LC cards, types and i 

sources of numbers, main entries, sending or- \ 
dars, checking in, follow-ups* j 

Approximate Time Required; Six to eight hours * 

• ■ 

Prerequisite Knowledge : Basic familiarity with a library and j 

with seme library terms (Farber and 
Elrod's programs) 

When and Where Field Tested : By 3 library employees at Kenyon 

and 2 at Ohio Wesleyan 

i 

i 

Results of Field Test : Corrected errors. j 

HUMAN NATURE 

Clarence Leuba, Department of Psychology 

Wright State Campus i 

1 

j General Objectives ; To enable the student to answer certain ; 

I specific questions regarding v/hat human | 

I nature is and is not, and regarding the | 

1 characteristics making up human nature. ( 

i I 

j Cours'.e for which Intended : Introductory or Educational Psychol-| 

1 ogy. Social Psychology | 

I ■ i 

I Topics Covered : Meaning of Human Mature, Innate aspects cr raw j 

! materials of human nature, Acquired aspects of | 

i human nature. Human nature as a descriptive and: 

I not an "explanatory concept. Human nature as 1 

changing, Original human nature. The concept of* 

the "natural man". I 

■ ! 

Approximate Time Required : Eight to ten hours 




f 



i 



Prerequisite Knowledge ; College level ability 

■^hen and Where Field Tested ; Earlham, Intro. Psych, classes 

DePauw ft Intro , Psych, classes 
Antioch, Intro. Psych, classes 

Ik 

Results of Field Tests : Revised program to present status 



THE SOIL: A PROGRAMMED TEXT 

W. M. Lotkowski, Department of Earth Sciences j 

Antioch College, Yellow Springs, Ohio . ! 

• I 

Genergi^l Ob j ec t ives : To enable student to answer such questions 

as: what is soil and how is it formed? j 
how do temperature, humidity, compositioi^ 

^ of the underlying rock material, vegeta- j 

tion, topography, time, and land use I 

affect soil? of what use are particular | 
soils, and how can they be used most 
effectively? 



fi.9*^t*ses for which Intended: Geography, conservation | 

^ . ■ * ' ^ ? ( 

Topics Cpvqred : Physical states of soil components; texture, j 

structure and water retention capacity; ! 

mineral composition and humus content; soil | 
processes; eluviation, illuviation, j 
and leaching. 1 

i 

Approximate Time Required : j 

■^yQ'^Qq.hi-site knowledge : For students brighter than average, j 

generally able to grasp concepts and ! 
retain information easily. | 

and Where Field Tested : Antiroh College '65 Office of j 

Program Development and Research! 
in Education 



I Results of Field Test ; Revised Program 

t 

i FORMAL STRUCTURE OF THE SHORT STORY 
I Kenneth B. Marshall, Department of English 
! Denison University, Granvil.le, Ohio 



i 

li 



General ObiectivA«t 



To train students tto 
within a short piece 
tain formal elements 
character, narrative 
attitude (including 



perceive function, | 
of fiction, of cer- j 
of structure: plot, I 
technique and | 

irony and use of sym- 



^!l 

i,l 

i 

■} 



I'- N 

I '’ l!‘ ' 

li->i 




• 



m 






Courses for which Intended; 



Introductory course in study of 
literature, including techniques 
of fiction or of literary forms 
required. Ability to read care- 
fully. Best for use with college 
freshmen. 



.1 i J|IW~l«fc-U r-^ 



When and Where Field Tested ; 
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Results of Field Test; 



PERSONNEL SELECTION; A SELF-INSTRUCTIONAL PROGRAM 
F. S. McKenna, Department of Psychology 
DePauw University, Greencastle, Indiana 



General Objectives ; To provide a self instructional and self 

containedunitonthe fundamental con- 
cepts and techniques of personnel selec- 
tion. Upon completion of this program, 
the student would be expected to be able 
to read the professional literature in 
this field with understanding, and to 
develop systematic personnel selection 
procedures . 



Courses for which Intended ; Psychology of Business and Industijy, 

Industrial Psychology, Personnel | 
Psychology, Per'^bnnel Administra- | 



t ion . 



Topics Covered ; Performance criteria, predictors, validation 

of selection procedures, and development of 
selection procedures. 



Approximate Time Required ; Four to five hours. 



Prerequisite Knowledge ; Designed for college underclass | 

students. Previous course in psychol-t 



ogy helpful but not necessary. 



I 



When ,and Where Field Tested; Wabash College, '64 



Ohio Wesleyan, '64 
DePauw University, '64 
Revision at DePauw, '65 



Results of Field Test ; Used for itew analysis and overall 

evaluation 



THE LITERARY RELATIONSHIPS AMONG MATTHEW, MARK, AND LUKE 
R. M. Md«*tg^^ Department- of Religion 
Ohio Wesleyan University, Delaware, Ohio 
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General Object.lves : Having completed the unit and given 

material selected from Matthew, Mark, and 
. Luke, the student should be able to ident- 
ify the evidence supporting the theory 
_ that (a) Matthew and Luke used Mark, and 
(b) Matthew and Luke did not depend upon 
e£ich other but upon an unknown source in 
their material which does not come from 
Mark. 

Courses for which Intended ; Bible, New Testament, Studies in 

Gospels ■ 

Topics Covered : The two source theory about the literary re- I 

lationships among Matthew, Mark, and Luke. | 

Approximate Time Required .- Two to six hours 



J 



Prerequisite Knowledge ; No previous knowledge of gospels re 

qnired; designed for students with 
verbal skill rated at above 450 on th 
ACE. 

When and Where Field Tasted ; Denison University, '65--Scott 

Ohio Wesleyan fipiversity, '65-~ j 
Vulgamc-re^ Illinois Wesleyan, *65 



Results of Field Test ; Results sent to the denter. 

An introduction to systematic analysis of POLITICAL SCIENCE 
Lois M. Pelekoudas, Department of Political Science 
’Antioch College, Yellow Springs, Ohio 




h 






General Objectives : To identify and to state a problem in 

political science, identify hypotheses and 
assumptions, to frame hypotheses, and to ' 
identify major approaches in works of ' 
political science. 

" j 

Courses for which Intended ; Introduction to Political Science ) 
Topics Covered ; See General Objectives 



\ 



.Approximate Time Required: About 9 hours 



N 



Prerequisite Knowledge : College level freshmen, cursory ]<n6 

> ledge, of the yocabuiary of American 
government • 



When and Where Field Tested : 



j : 






it 

ll 
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THE LANGUAGE OF LOGIC 

Morton Schagrin, Department of Physical Science 
Denison University, Granville, Ohio 

General Objectives ; To prepare students with no background in 

modern logic with sufficient familiarity 
to reed with comprehension recent studies | 
in the philosophy of science and semantic^ 

which are written in this notation. i 

! 

which Intended . Philosophy of Science, Sem*antics, 

Epistemology .and could help 
some students in symbolic logic- > 

IP-pics Covered; Interpreting and Symbolizing in the sentenciaJ 

and first order predicate calculus with 
identity. Class abstraction. 

Approximate Time Required ; 6 1/2-7 hours 

? Q q u i s i t e Kno w ledge ; College Freshmen with no previous 

knowledge. 

■ — When and yhere Field Tested ; Nabash College Fall '64 Hackstaff 

Denison University Fall *64 
Schagrin > — 

HEARING MUSIC WITH UNDERSTANDING 
Paul Schwartz, Department of Musi:: 

; Kenyon College, Gambier, Ohio 

I 

I ? An introduction to elements of music | 

I through sight and sound; a text, consist- | 

^*^6 9 chapters, with tapes accompanying^ 

each chapter, j 

Q9_^^^s e8 for jwhich Intended ; Introductory music courses, fine { 

arts survey courses, usually I 

offered without a prerequisite in 
music. Also as a review of musi- 
cal elements preparing for a more 

I advanced course in music theory. 

' P ^ ° -Coyer e d : Time and pitch; meter and rhythm; intervals, 

; scales and chords; basic homophonic and 

•V polyphonic form elements* 

App^^oxi mate Time Required ; One month (at 2 Chapters per week) 

Pyg^ogusit e Knowledge ; None, a program for beginners; no 
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previous knowledge required. > 

When and Where Field Tested ; - Jeanette Sexton, Music Education, I 

School of Music, Ohio State Univer- ; 
sity - Charles R. Hoffer, Director | 
of Music, Board of Education, Clay- \ 
ton, Missouri - Paul Schwartz, 
Chairman, Department of Music, Ken- 1 
yon College. ; 

• i 

_of- Test ; Suitable for three grade levels: ' 

1 ) High school juniors and seniors * ‘ 

2) Undergraduates ; 

3) Graduate students without previous musi- ^ 

cal experience. . 



MUSIC: BEGINNING EAR-TRAINING 

L. R. Smith, H. B. Ray, R. A. Haminar, Department of Music, ‘ 

Kalamazoo College, Kalamazoo, Michigan I 

i 

General O bjectives ; To develop basic listening skills (Per- | 

ception of intervals and rhythmic 
Patterns.) : : 



Cputses for which Intended : Beginning Theory : 

. I 

— : Intervals in the major scale; rhythia patterns- 

in simple meters; intervals in minor; / 

rhythms in compound | 

0 

f 

APP^o ^iniate Time Required : One week to six weeks 1 

- i 

Pr^erequisite Knowledge ; Understanding of music notation * 



g n a n d Where Field Tested : Albion College, Winter, 1965 

Results Of Field Test : Generally favorable. i 

BIOCHEMISTRY FOR BIOLOGISTS | 

William K.’ Stephenson , Department of Biology, [ 

Earlham College, Richmond, Indiana 1 

i 

. . . I 

Objectives : Students will attain the chemistry and ! 

^biochemistry requisite for the first 
course in contemporary college biology. | 

fot* which Intended ; General biology courses, also * 

.cf^plicable to ; gene t ics and -- | 

' physiology. 






Topics Covered : 

Unit 1. Atoms and molecules Unit 8. Amino Acids 



2. Chemical groups 

3. Ions, salts } crystals 

4. Bonds 

5. Equilibrium 

6. Carbohydrates I 

7. Carbohydrates II 

Supplementary Units 
Unit S-1 Molecular Weight 
S»2 p H 



9. Proteins I [ 

10. Proteins II | 

11. Fats, Phospholipids, Sterbls 

12. Muclear Acids 

13. Energy | 

14. Biochemical correlations 1 

1 

Three Review Units ' *' 



Approximate Time Required ; Average 12 hours, range 6-22 hours.! 



Prerequisite Knowledge ; 1) No previous training in chemistry ! 

required. ‘ 

2) Competence in arithmetic computation 
and in simple algebraic manipulation 

3) Average college level entrance abilf 
ity to read. 

i 

When and Where Field Tested ; William K. Stephenson Earlham j 

College, Fall, 1964. | 

Francis Yow, Kenyon College, Falli 
1954 ■ J 

, Donald Smith, Ohio Wesleyan Uni- | 

versity. Fall, 1964. | 



Results of Field Test : Used in revising program, 

I* 

IsTUDY HABITS 

jF, P* Van Eyl, Department of Psychology, 
jHope College, Holland, Michigan 

I . 

} General Objectives ; To develop study habits that lead to a 

; more successful way of studying textbooks 

I Courses for which Intended: All 



\ ■ 



Topics Covered : Study habits 
Approximate Time Required : 

Prerequisite Knowledge : High'School, college entering student 
When and Where, Field Tested; Fall ’ 64--Earlham--McDowell 



I 



Fall '64--0hio Wesleyan--Whitted | 




■ 



I 



0 

m 
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PUBLIC SCHOOL FINANCE 

J-« No Wetmore, Department of Education, 

Ohio Wesleyan University, Delaware, Ohio 

! 

G^eral Objectives : To teach undergraduate students all as- 
pects of public school f inance--local , 
state and federal 

^,^ich Intended ; Education-School Administration 

^ QP ^ c s C o V e r e d ; Public School Finance: local, state, and 

federal (Program not completed.) 

Prerequisite Knowledge : None 

When and Where Field Tested ; DePauw University 

THE NATURE AND USES OF COLOR 
Forbes Whiteside, Department of Art, 

Oberlin College, Oberlin, Ohio 

Q^jsctives : To help th.e student gain familiarity, with 
i the interc\c t ions between hue, value and 

intensity 

‘ fpT which Intended: Introduction to Art Studio Course 

r 

loRjigs Covered ; Hue, Value, Intensity of relationships 
I Prerequisite Knowledge : None 

i When and Where Field Tested : Studio students at Oberlin 

; College 

BIRD SONGS 

James B. Cope, Department' of Biology, 

Earlham College, Richmond, Indiana 

i 

^Qtieral Objectives : To teach students bird songs with the use 

of audio and visual aids. 

I Cj^urses for which Intended : Vertebrate Zoology II 

* ■■ 

I Topics Covered : Ornithology 

I , , , ... 

I Approiiimate Time Required : 

Prerequisite Knowledge : None 
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j Where and When Field Tested : Earlham Students | 

^ Albion College j 

Hope College 

Results of Field Test : No report | 

ORGANIZATION OF THE LIBRARY ! 

Evan Farber, Library ■ 

Earlham College, Richmond, Indiana ' | 

General Objectives : To train subprofessional help in .library ■ 

organization and entries. . I 

Topics Covered : 1) Organisation of library 

2) Unit card preparation j 

3) Ordering of LC Cards 

■*^4) Continuous adding j 

5) Films 1 

5 

Prerequisite Knowledge ; High school education. 

* 1 

A PROGRAM IN COMPOSITION j 

s Fred L. Bergmann, Department of English ■ 

; DePauw University, Greencastle, Indiana j 

■ General Ol^jectives : To enable the student to recognize and to; 

! write effective paragraph's through recog-l 

nition of the basic principles of unity, ; 
coherence, and emphasis. i 

* Coui^ses for which Intended : College freshman English; ad- j 

vanced sections of fourth-year high 
school English; self-instruction.' 

Topics Covered : Unity, coherence, emphasis ; 

; A pproximate Time Required ; About 3 weeks in the classroom; i 

i about 10 hours individually. I 

'■ ■ ' - ‘ . . * 

I Prerequisite Knowledge : Upperlevel high school seniors, •) 

{ college freshmen. I 

' I 

When and Where Field Tested ; First semester ’65- *''6 DePauw-- \ 

;■ _ ■ . - ’ Pence \ 

\ , Ohio Wesleyan--Whitted 

Kalamazoo--Roerecke i 

' ' ■' ^ t ■ ■■ j 

Results and Where Field Tested ; Each field tester reported J 

improvement in student writing. 
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FACT AND FORM: ELEMENTARY PRINCIPLES OF SENTENCE DESIGN* j 

Owen Duston, Department of English, 

Wabash College, Crawfordsville , Indiana 

Q_eneral Objectives ; To enable the student to use the re- 
sources of sentence structure to or- | 

ganize the facts of his experience. ' 

II 

Courses for which Intended ; The introductory phase of com- 
position courses. | 

Topic s Covered; The use of and ; of subordination with ‘when I 

and while ; of the descriptive clause wit h | 
who , which ,^and that; of the participial ! 
phrase; of the noun clause with that. 



AP.P^Qxin^^’ts Time Required ; Three or four hours . j 

Pygyeguisite Knowledge ; It may take some tTm‘e to determine at! 

what level the program will be most ! 
appropriate. « 



When and Where Field Tested ; Not yet field tested. 



A PROGRAMED INTRODUCTION TO SARTRE'S ANALYSIS OF FREEDOM ! 

Robex-'t Fichter, Department of Religion { 

Ohio Wesleyan University, Delaware, Ohio j 

t 

Gener al Objectives ; (a) Find the rule which governs talk abou*^ 

Freedom j 

(b) Apply the rule to talk about Freedom, i 



C oyyses for which. Intended; Philosophy, religion, humanities, 

social studies 

Topics Covered; Freedom 



Approximate Time Required ; 

Py ® ^ ^sito Knowledge ; Genera., knowledge at college level 

Where and W hen Field Tested ; Program has not reached this 

stage. 



PASO A PASO, A BASIC COURSE IN SPANISH THROUGH MEANING AND STRUCTURE 
Renato J. Gonsales, Department of Spanish, 1 

Albion College, Albion, Michigan I 

i 

general Objectives ; To teach introductory . Spanish ; more | 

specifically to train the student in the ! 
basic elements of Spanish by directly j 
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interacting with the languages without re- 
course to traditional grammar or transla- 
tion skills. 



Topics Covered ; The pedagogical elements of the course consist 

primarily of 1) habit forming methods 

--repetition 
--substitution 
--question and answers 
2) structure 

All structure is taught not by means of tra- ! 
ditional grammar terms but by utilizing* ] 

"structural linguistics" to analyze and . i 

\ d<r 

c> 



synthesize 

^Prerequisite Knowledge ; Entering college freshm 



ALTERNATIVE LOGICS 

L. H. Hackstaff, Department of Philosophy 
Wabash College* Crawf ordsville , Indiana 



general Objectives ; To teach undergraduate students sevel | 

alternative systems of propositional logici 



Courses for which Intended ; Elementary and/o:^ Intermediate 

Logic. 



^opics Covered ? See General Objectives 
A pproximate Time Required ; 1 1/2-2 weeks 



Prerequisite Knowledge ? Student must have studied sections 6- 

10 of F , B. Fitch’s Symbolic Logic 



When and Where Field Tested ; L. H. Hackstaff, Wabash Fall '65 

L. H. Hackstaff, San Francisco 
State '66 



LABORATORY SUPPLEMENT FOR CHEMISTRY 11 and 12 

Alfred Henderson and Laurence Strong, Department of Chemistry 

Earlham College, Richmond, Indiana 



General Objectives ; To Introduce students to an experimental 

study of interaction among the compounds 
3 mixture. The student sees how con- 
tinuous variation experiments can provide 
quantitative data suitable for establish- . 
ing chemical equations. He also learns to j 
test the humidity of proposed chemical I 
equation by designing his own continuous 
variation experiment. | 
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^s^cription of the Program; This programed study included the 

pr eparat ion and use of quantita- 
tive solutions in terms of mass 9 
volume and moles. It includes 
colored photographic slides keyed ‘ 
to tea''hing frames of the program^ 
The study moves from a general | 
view of interaction to the particj 
ular situations that are defined j 
as chemical reactions. Chemical 
reactions are thus defined in 1 
straight-forward operational terms. 
The operational approach proceeds 
directly to the dtoichiometry ' it ‘ is 
then possible to develop a chemi-H^ 
cal equation as an interpretation 
of th^ reaction. Graphical inter-* 
pretation of the data is em- ! 

phasized. | 

P^rerequislte Knowledge> ^For use in individual instruction in ! 

conjunction with re^Ulsr lecture and | 
laboratory instruction for first year 1 
college chemistry. , j 

V I 

MUSICAL RHYTHM (METER, PARTS A, B, AND C) i 

Leonard Holvik, Department of Music, j 

Earlham College, Richmond, Indiana j 

General Objectives ; To bring students to a common level for I 

discussion of rhythm and related matters i 
in the c.lassroom--to give them command of | 
the verbal material and concepts and of 
the actual musical phenomena concerned. * 

I 

Cour ses for which Inteuded ? Introductory courses for general 

students commonly called* Intro, to 
Music or Music Appreciation • 

Topic s Covered ; The metrical basis of common practice rhythm, | 

the notation of rhythm, and the subject of ; 
time signatures and related matters. 1 

j 

Approximate Time Required e Three to Four hours | 

I 

Pye y e q » i S i t e Knowledge ; None except that for college admission 

I 

When and Where Field Tested : Kenyon, Paul Schwartz ; 

Albion, Doran, Obetz, Riseling 
Earlham, Holvik ! 



THE ECOLOGY OF PUBLIC ADMINISTRATION 

William L* Morrow, Department of Political Science ! 

DePauw University, Greencastle, Indiana ; 

After completing the program, it is hoped that the student: 

A. will be able to appropriately recognize and compare the 
major controversies and concepts surrounding the study of i 
administration. 

B. will be able to demonatrate satisfactorily the "multi- ’ 

discipline" character of administration, and: 

C. will possess an appreciation of the nuar-ces and finer points 

of meaning and interpretation of all the different { 

phenomena presented. ‘ 

Satisfactory performance will be measured through an essay examina- 
tion. 



General Objectives : To help the beginning student of public ' 

administration attain a realistic perspec-: 
tive of the nature of administration in 
general. 

i 

Courses for which Intended : The introductory course in Public i 

administration » 

I 

Topics Covered : The problem of conceptualiza|:ion ; the ecology 

of administration; administi^f tion as an inter-j 
disciplinary study; administrative theory, ad-> 
minis 'rration as Science; administration and I 
politics. 




Approximate 

Prerequisite 



Time Required: 3-4 hours ■ 

Knowledge : Aimed at college juniors. Perfor- 
mance would probably be better from 
those who have had introductory courses 
in sociology, psychology, and govern- 
ment. 



W hen and Where Field Tested : Thus far only field tested in own' 

classes. 

Results of Field Test : Results have been fairly successful. 

! Revision still necessary. 

INTRODUCTION TO MODERN PHYSICS 

B. R. Russell, Department of Physics ; 

College of Wooster, Wooster, Ohio 

1 

General Objectives : To present certain key topics in modern 



physics in a form suitable for use as 
supplementary material in*a beginning 
college physics course 

Courses for which Intended ; Elementary College Physics 

Topics Covered ; Rutherford Scattering, Bohr Theory of the 

Atom, Quantum Concepts 

Approximate Time Rt^quired ; 6 hours 

Prerequisite Knowledge ; Mechanics, Electricity, and Optics 

(Calculus not required.) Average 
students j 

I 

When and Where Field Tested ; Kenyon, April '66--Miller 

Albion, April * 66-“Glathart 
Wooster, April * 66--Stephenson 

A PROGRAMMED REFERENCE GRAMMAR FOR ELEMENTARY SPOKEN AND WRITTEN 

FRENCH 

Richard R. Strawn, Department of Romance Languages ^ 

Wabash College, Crawfordsville , Indiana 



General Objectives ; To state the grammar rul^s for each topic | 

treated and to apply thq rules correctly,’ 

on paper, to new instancies. ! 

; , I 

Courses for which Intended ; First and second semester French ‘ 

“ f 

« I 

■ . 9 

; Topics Covered ; Gender, designation, de-relationships, ad- > 

j jective agreement, tense (Periphrastic future 

! present, passe compoje,) interrogation (yes- ! 

no, questions, defining and distinguishing, I 

! written plurals of nouns and adjectives. ; 

i 

Approximate Time Required: 10 hours I 

i i 

Prerequisite Knowledge : Ability to discriminate one phonetic 
! symbol from another. 

I i 

j When and Where Field Tested : Wabash College, January, Feb. *66 

‘ Celler, Strawn I 

, CRYSTALS: AN INTRODUCTION — A PROGRAMED LABORATORY AND INDEPENDENT 
STUDY UNIT 

John F. White, Department of Earth Sciences 
Afitioch College, Yellow Springs, Ohio 



General Objectives : 1) to provide a stimulating and efficient 

programed study unit on crystals for 



introductory courses 

2) to present the ma^*erial so it can be 
useful for both science and non-science 
students 

3) to provide for the integration of ma- 
terial that is ordinarily presented 
separately through lectures, text, and 
laboratory. 



Courses for which Intended : As auxiliary material for:. Intro 

Geology, Earth Sciences, Chemistry-, 
Materials Science, Physical 
Science, crystallography. Mineral- 
ogy, Metallurgy. 

Topics Covered : Crystals and Solids, History of Crystal Study,! 

Lattice and Structure, Unit Cell Content, Rel-» 
ative Size of Atoms, Theoretical Density, ? 

Isomorphism and Polymorphism, Solid Solution | 
and Imperfect Crystals, Symmetry, Review and I 
Additional Structural Models. j 

Approximate Time Required : 5 hours » 

Prerequisite Knowledge : Ability level--first year college | 

When and Wiyere Field Tested : Antioch College, Fall *65--White I 

Results of Field Test : So.ne revision and supj^orting frames I 

required* Average score 89% I 



ELECTROPHYSIOLOGY OF NERVOUS TISSUE 

Francis W. Yow, Department of Biology j 

Kenyon College, Gambler, Ohio j 

I 

: Use and theory of Stimulator-Oscilloscope 
Complex and interpretation of Oscilloscope 
Image with respect to transmition of nerve] 
impulse* | 



,r General Objectives 

A 



Description of the Program: 1) 



2 ) 

3 ) 



i\ 

If 



8mm film loop showing nerve f 
preparation and instrument com-! 
plex, along with oscilloscope \ 
trace, 1 

A brief program introducing thei 
student to oscilloscope and I 
interpretation of trace, | 

tape to be fed into oscilloscope 
which will reproduce nerve re- 
sponse as well as addio des- | 
cription of events* 
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Sample Pages from Selected Prot^rams 



iiaftu pages from nine programs produced in both 

’^5® samples are taken from programs that 

...? ased in the intensive evaluation made under a variety of con>» 

Chapter 6. The remaining four were selected so 
as to illustrate the variety of programs that were produced on the 
.project e 



67 






I- 






■ . \ . 






1. Sample page from POETRY, METHOD AND 
‘MEANING, by James Cook 

! 



36 

Images constitute another very important 
class of figures of speech. One might say 
that an image results when a passage Is so 
vividly descriptive that the reader 
imagines a sensory experience. For example: 
”The smooth and creamy vanilla ice cream, 
cool upon my tongue, melted gently toward 
my tonsils.” Now, while you really can't 
taste the ice cream nor feel its texture 
or its temperature, you can imagine that 
taste and that feeling. 



Try it I 



Got it? 



Yum! 



Thus, an image if a figure of speech that 
.represents a coi|i|rete experience or an * 
.object by appealing to the senses through 
the * 



51. 



A poet may appeal to sight, boaring, smell ^ 
taste, touch, the. senses which distinguish 
changes in temperature, which register 
balance and motion, and which register 
visceral reactions. For our purposes, in-| 
elude the last three under touch. j 
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1. (Cont) Sample page from POETRY, METHOD Al»D MEAKIN6, by Jamas CooJc 
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37 



Which of the following quotations contain images? Circle the 
appropriate letters. 



A. 



All in a hot and copper sky 
The bloody sun at noon 
Right up above the mast did stand 
No bigger than the moon. 



i. 



Grow old along with me. 
The bast is yet to be. 



C. 



I cannot see what flowers ar^ at my feet. 

Nor what soft incense hangs upon the boug)is. 

But, ip embalmed darkness, guess each sweet 
Wherewith the seasonable month endows 
The gbass, the thicket, and the fruit trap wildj 
White hawthorn, and the pastoral eglantine. 

Fast fading violets cover’d up in leaves -ir 
And mid-May* s eldest child. 

The coming musk-rose, full of dewy wine, 

The murmurous hsunt of flies on summer eves. 



f 

Observe that the tied images in A aad**C- are such that they I 
virtually the same imaginative responses in most readers! 
Look back at A and C and analyze your responses. To which {' 
senses do the tied images in A and C appeal? (They may appeal | 

to more than one. ) List them. Which words seem to control ■* 
that appeal? [ 



Senses 



Words 









1. (Cont.) Samplo page from POETRY, METHOD AF MEANING, by James 

Cook 



38 

Your list will probably be something like this one. You may have 
others, but if you missed these, look back. 



-I 




Senses 



Words 



A. touch 
sight 



simultaneously 
, stimulated 



hot and copper 
bloody sun at noon 
above mast 
no bigger than moon 












a 

Cl 

I 






# 
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B. smell 

smell and touch 
smell 



taste 



hearing 



■ ' 0' a' ■'.a . 

.! - i. ii'P' P C «; C v ■. --i u 

'?■ .V t IV ;■ .V t * 












flowers at feet 

embalmed darkness 

grass, thicket, fruit treses, 
white hsTfthorh, eglantine, 
violets, musk*rose 



dewy wine 



murmurous flies 



!v , 
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2. Sample Page from LANGUAGE OF LOGIC, by Morton Schagrin 4 

^ English sentences can be separated into those that are simple 
(or atomic), and those that are compound (or molecular). 

SIMPLE SENTENCES 

a) Small grey doves do coo al lonesome lovers. 

1 

lb) The com is <^uite tall. 

* • 

; COMPOUND SENTENCES 

ia) Dogs lope while horses gallop. 

;b) If the balance of terror persists, then the rate of cigarette 
I smoking increases. 

! 

Roughly speaking, compound sentences are composed of two or more 

shorter simple sentences. Circle the compound sentences in this 
list. 




The quick brown fox jumped over the lazy dogs. 



12) Roses are red ®nd violets are blue. 

i ’ 

■3) Now is the time fcr all good men to come to tjjf aid of the party 

|4) Fat burns, but water doesn*t (burn). 

|5) He who never speaks never errs. 

( 

j6) If a man never errs, then the Devil isn*t hippy. 

|7) Simple sentences, standing alone are easy to detect, nevertheless 
I It is often difficult to analyze a compound sentence into its 
! component simple parts. 



•8) Seldom have so many owed so much to so few. 

i 

i 

j- » -ANSWERS ------- -ANSWERS 

(t 

j2) Roses are red and violets are blue. 

|4) Fat burns, but water doesn*t. 

• 6_) If a man never -errs, then the Devil isn*t happy. 

|7) Simple sentences standing alone are easy to detect, nevertheless, 
i it is often difficult to analyze a compound sentence into its 
I component simple parts. 



If you are coi:^ect on all of these, turn to 

“ ‘ ■' ■ page 7 

If you have missed any of these, turn to 











OK 



2 , (Cont.) Saiiiple page from LANGUAGE OF LOGIC • by Morton Scbagr^n 
You missed one or more compound sentences. 

! i ♦ — 

I Perhaps you should concentrate onj identifying simple sentences. ' 

, Simple sentences 'have not shorter I sentences contained within them. 

1 

! Example ! DOGS BARK. 

: Now modifiers are irrelevant to the logical simplicity of a sentenc4* 

fi 

; Example! Small, shorthaired black dogs often bark fiercely.. 

I This is still a simple sentence; 'Some thing (the subject) does some* 
j thing (the predicate),. 

5 

! Were we to compound Dogs bark" with another simple sentence, 

; we might obtain something like!, 

1 If dogs bark , then cats purr . 

I 

I Circle tl)e compound sentences below! 

j 1) That doctor is a fraud or this druggist is a charlatan. 

I 

I 2) That child kicked this child in the stomach. 

I 

I 3) Aversive reinforcement is rewarding to a masochist. 

i 

I * 

,| 4 ) I will not run if I am nominated, and I shall not serve if I 
am elected. 



ANSWERS 



•ANSWERS 



, 1) That doctoi? is a fraud or this druggist is a charlatan. 

{ 

i * . * 

1 4) I will not run if I am nominated, and I shall not serve if I 
1 am. elected. * 



If you are incorrect, go to page 13. 
If you are correct, go to page 7. 
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3. Sample page from BIOCHEMISTRY FOR BIOLOGISTS* by Millla® 

Steptssatison 

(29) 



tirAt>AAAilltAeAAil1ttnrtnrtnreinttnrtllr*A*#rAAAAillrA<kAA*#( 

I 
# 

; 6. What is the general characteristic of the ions in the preceding! 
item (when compared to the atoms)? 



' T*»e ions have gained or lost electrons and bear a charge, (or 
! similar response) 

7, The sodium ion is written Na+. Ca ion = Ca+t 
Write the symbol for the Mg ion. 

; Write the symbol for the K ion. 



Mg++ and K+ 

pV A ft ft A A * A A )lb A A ^ ^ ^ A A ft ft ft ft ft ft ft ft ft ft 

I 

I 8 . 



Outer Shell 
Diagram 



•# 



•a./' 



f • 



How many electrons can Cl accept to form an 
(complete the outer electron shell)? 

Write the symbol for the Cl ion. 



ion 



J 



t 

( 

i 

■ ft ft 
i 9 . 



One 

Cl- ' , 

ftftftftftftftftftftftftftftftftftft Aftftftftftftftftftftftft I 
Write the symbol for each of the following ions: 

Ma Ca F 

Cl K ~ 

Mg 



Na Nat 



Ca Ca++ 



Cl C 1 — 



K 



K.+ 



! ft ft 



Mg Mgtt 

AAAftftftftAAAAftftftAft. AAAAAAAAAftAftftft 

.i. n 



A a| 

<g.'».i-pw.n'iii • j[rii 




















m 






=er|c 






k K ■' V«!r<i™« 



^t:».-isi^^A- 1.:*-'*' r ” 
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3. (Cont.) Sample page from BIOCHEMISTRY FOR BIOLOGISTS* by 
William Stephenson 

(30) 

Cation = + ion pronoynced - CAT' ION 

Anion = - ion 

Pronounce cation and anion out loud £ times each . 

Label each as a cation or anion. ~ 

• Na Ca F 



Cl 

Mg' 



K 



Na cation 



Cl anion 



Ca cation 
k cation 



anion 



; Mg cation 

11. In the two dimensional diagram (Panel B), how many cl- are ad- 
jacent to (bonded to) each Na+? 



‘ four 

‘AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAftAAftAAAAAAAAAAAAA 
' 9. With the two dimensional diagram of Panel B* construct a mental 
■picture of the third dimension in a crystal. Remember that Na+ and 
*C1- alternate in each dimension. Now, in the crystal, how many Cl- 
jare adjacent to ^ach Na+? How many Na+ are adjaceiit to each C1-? 

I 6 ! 

I ^ 

6 . 



jAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA|tijkAAAAAAAftAAAAAAA 

>12. A salt is a general term for an ioni c compound. Of yhat uni 
{is a salt composed? 



ts 



1 - 



I ions 

'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 



13. Most salts have metallic (Eg. Sodium, Potassium, Magnesium, 
■Iron, Calcium, etc.) cations. 



'Circle the metallic ion in each of these salts. 
I + - 



NaCl 









( 

i 

f 

\ 

a 

i 

i 

U» M. « 


++ - 
CaF 

++ 2 - 
FeClg 




• 




0" 

♦ 


Cl- (( 




©"Cl2 




1 

i 

! 
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Sample page from LITERARY RELATIONSHIPS AMONG MATTHEW, 
MARK AMD LUKE by Robert Montgomery 



FRAME # 2 



The passages from the last frame are given you again with the 



words underlined where the three Gospels agree, in order that you 



ican answer another question about the degree to which the Gospels 



'correspond 



.he asked his dis- 



*ciples, ” Who do men 
■say that the Son of 



Man is?” And they 
said, "Some say John 
the Baptist , others 
say Elijah and othtra 
Jeremiah or one of 
the prophets .** 

|. . . " But who do you 
say that I am?" Si- 

wmmmmmmmmtmmm mmm mmmmmm 



mon Peter replied, 
f'You are the Christ > 
!the Son of the 
kiiving God." 



his dis- 
ciples, " Who do men 
say that I am?" And 
they told him, " John 
the Baptist ; and ot - 
hers say , Elijah , and 
others one of the 
prophets .** 

• • • do you 

El ay that ^ am?" Peter 
answered him, "You 
are the Christ . " 



... he asked them 



" Who do the people 
say that I am?" And 
they answered, ** Joh n 
the Baptist ; but 



others say, Elijahs 
anT o’there that one 
of the old p rophets ' 
has risen." 

• • ' ^ho do you I 

say that J[ am?" Petet" 



aneifered "The Christ 
of^^God." 



Q, • 



Check statements which correctly describe other agreements to 
be found in the passages above. 



A sentence begins with "an'*." 



b. 



Exclamations occur in the same line as is shown by an 
exclamation mark. 



Questions, rather than declarative sentences, occur at 



the same point. 



f- 



Considering both the sharing of significant terms and trivial 
stylistic matters such as the use of "and," the three Gospels 
(do) (do not) display a significant amount of literary corres- 
pondence. 



ANSWER 



jl . 



a. and c. 



2 . 



do 



Is the agreement displayed above substantially more than one would 



.expect among three who tell abou1^“~the same event? 



■7‘sr — 


















4. (Cent.) Sample page from LITERARY RELATIONSHIPS AMONG MATTHEW,' 

MARK AND LUKE, by Robert Montgomery 

FRAME # 3 



jl. For example-, since the death of Jesus on the cross is a sacred 
I event to Christianity, as a matter of course one (would not) 

' (would) expect each early Christian account of Jesus* life to 
, give a majority of his last words on the cross > 

I 

« To see what the situation is with respect to the last, words 

1 of Jesus, inspect the passages below which give the last words of 
I Jesus • 



Matthew Mark Luke John 



"Father, forgive 
them; for they know 
not what they do." 



"Woman, behold you? 
sonJ " 

"Behold your 
mother! " 

"I this’st." 



J 



1 

I "Eli, Eli lama 
] sabachthani?" 

. that is , 

i "My God, my God, why 
; hast thou forsaken 
* me?" 



"Truly, I say to 
you, today you will 
be with me in 
Paradise . " 

"Eloi, Eloi, lama 
sabachthani?" 
which means, 

"My God, my God, why 
hast thou forsaken 
me?" 



"Father, into thy 
hands I commit my 
spirit." 

"It is finished." 



I 2, (Matthew)(Luke ) and (Mark)(John) agree on (1)(2)(3) of the 
t utterances of Jesus on. the cross. 



3. If with regard to the very ^-as^t expressions of Jesus , the 

situation is as it standti above we (would not)(would) expect 
that there would be much agreement among Matthew, Mark, and Luki^ 
about the other sayings of Jesus. . . 



Is it an exception, therefore, to find the amount of agree- 
ment among Matthew, Mark, and Luke which we discovered in the first 



frame? 



16 












-J 



M 
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5. Sample page from AN INTRODUCTION TO SYSTEMATIC ANALYSIS OF 
POLITICAL SCIENCE, by Lois Pelekoudas 



24 



S.’ 



i Place an x in the appropriate blank to indicate whether a problem 
I for inquiry is explicitly posed in each of the following paragraphs 

I If a problem is explicitly stated, rephrase it in the space pro- 
ivided or underline it in the text of the paragraph. 



V' 



>-~ ^If n o problem is explicitly stated, rewrite the paragraph so that 
I it df^es pose a problem for political inquiry. 



1 . 



There is no provision in the U. S. Constitution for a cabinet% 
Federal agencies have been formed as part of the executive 
branch, rather than the legislative branch of the government. 
Since the turn of the century, the number of federal regula- 
tory agencies has mushroomed. A problem for Inquiry is the 
combination of factors which have led to the formation of a 
federal ref^latory agpnoy and the type of structure the agency 
takes. 



9 



Problem 



No Problem 






AO 






%: 



2 . 



With few except! on sV^vil presidents who have sought second 
terms have been elected.^ One president served four consecu- 
tive terms, but such long tenure was banned by the Constitu- 
tional amendment (Amendment 22) added during the Truman term. 
The following study will examine the economic and political 
trends in the nation during those campaigns when an incumbent 
president failed in his bid for re-elections. 



Problem 



No Problem 
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S. (Cont.) Sample page from AN INTRODUCTION TO SYSTEHATIC ANAL- 
YSIS OF POLITICAL SCIENCE, by Lois Pelekoudaa 

48 

Fill in the blanks in the review paragraph below. 

Inquiry about a problem is guided by one or more , 

which must be relevant to the problem posed. The . 

are derived from a set of a which can be "chainec 

back*' to the most basic beliefs about the nature of knowledge, man. 
and the universe, and which have been formed on the basis of ex- 
perienced and observed phenomena. A set of related 
which immediately underlie and generate hypotheses constitutes a 

. It functions to and 

events, and if it does sO| it may be said 
to have utility, or to generate testable hypothes||f. 



i 



I hypotheses, hypotheses, assumptions, assumptions, theory, explain. 
I predict 




Y8 






6* 



A s 

4s 



I 



y 
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Sample page from HEARING HUSXC WITH UNDERSTANDING » by Paul 
Schwarts 



«»ri 



CHAPTER VI deals more fully WITH SCALES AND CHORDS. 



You discovered in CHAPTER IV that SCALES are formed 
by e^’ep*^ »rise mot ion t and in CHAPTER V that CHORDS 
are formed by wider INTERVALS than steps. In this 
chapter • you are invited to_find out roore specif icAl^(^ 
ly how a MAJOR SCALE is formed • what it looks like 
on paper • and how it sounds when played. 

You will be introduced to several SCALES which use the 
black keys of the keyboard. For this reason* it is es« 
sential that you remember clearly the functiops of the 
ACCIDENTAL^: the SHARP* the FLAT and the NATI|1RAL. By 
the same token* you should practice the new SpALE forma- 
tions on a keyboard instrument soon after you have worked 
your- way through this chapter. 



VI«A 



I 
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PRECEDING PAGE BLANK- NOT FILMED 



7. Sample page from A PROGRAM IN COMPOSITION, by Fred Bergmann 
II* Find the topic sentence in the following paragraph* 

Underline it* 



1 * 

2 * 

3 . 

4. 



The common cold is really a highly popular complaint 
throughout the temperate regions of the world, where it is 



endemic* It has all the attractions of an illness and none 



of the disadvantages, for it never kills anyone and always 
gets better within a fortnight. Meanwhile it acts as a, 

magnet for sympathy in midwinter, v?hen sympathy is needed 

» 

i 

the most 4 It grants a week's leave from the office at a 
time whep the holidays are miserably -balanced between re- 



h: : 



colleO.lc^n and anticipation* And as it iiiiiikes no difference 



to the .prognosis whether treatment is givdn or not, each 



}a\1 



cold offers its victims a delightful exerplse in self- 






doctoring* 

The topic sentence appears where in the paragraph? 
Is the paragraph unified?^ 



Is extraneous material brought in? 



Each of the sentences which follow the topic (gives a reason) 
(does not give a reason) for the popularity of the common 
cold* (Underline one*) 



Are the sentences which follow the topic in an acceptable 
order? 



81 
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7. (Conto) Sample page from A PROGRAM IN COMPOSITION* by 
Fred Bergmann 11. 



1. The topic sentence is the first sentence . 

2* Yes t the paragraph is unified* 

1 3, there is no extraneous material. 

1 4. Each following sentence gives a reason for the popularity of 
the cold. 

{ 

I 5. Yes i the following sentences are in acceptable order. 



I " ' - 

t 

' The paragraph on the common cold has complete i|nity and per- 

I f ect clarity. ^^|ie topic sentence at the beginninjjj iiialkes a state- 
i ment which call||- for reasons* Each sentence whicp fellows clari- 
jfies the topief itatement by giving one reason--ea|b- supplies part 
I of the answer to the reader *s inevitable quest ionf] :why ? Sentence 
1 2 indicates that the common cold has ail the attri|j,j|^'i6ns of an 
I illness and none of the disadvantages, as it never "tills the 
[patient. Now the reader must know what the attractions are. 

1 Sentefica 3 gives sympathy as a reason, sentence 4 suggests one’s 
fright to avoid work} sentence 5 adds the opportunity of harmless 
s self-doctorlng. These four sentences have expanded upon the topic 
at the same time that they have clarified it. 



ll'! ' 
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8. Sample page from A PROGRAMMED REFERENCE GRAMMAR FOR 

ELEMENTARY SPOKEN AND WRITTEN FRENCH, by Richard Strawn 



Cz 


ve k m se] 


Cz 


n 


ve p 


k m se3 


C z 


k m s] 


Cz 


n 


k m 


s p 3 


C z 


e k m se3 


[z 


n 


e p 


k m se3 



Our temporary rule in frame # 9.1 was too loose. It would let us 
come up with such a phrase as * [z n ve k m se p ], which is wrong 



TEMPORARY RULE. 

To negate a predicate, put [ 3 in front of the 

only) verb-word and l _. ] immediately after it. 



(or 



Cn( )3 



first 



Cp 3 



9.2 



[ va marse3 
[ mars3 
C n a marse3 



[va t marse3 
[mars t ] 

[a t marse3 



[n va t p marse3 
[n mars t p 3 
[n a t p inarse3 



(S) 



Evelt^ our temporary rule in frame # 9.2 is too loose. It would let 
us come up with such phrases as [n va p z marse3 or * 

[n va p t marse] , both of which are wrong. 

RULE. 

To negate a predicate, put [ 3 in front of the first (or 

only) verb-word. Put [ 3 as close after as it can come without 



C2) 

Cn( )3 Cp (z)3 

dividing a verb-plus-subject phrase 
(or equivalent answer) 
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8. (Cont.) Sample page from A PROGRAMMED REFERENCE GRAMMAR 
FOR ELEMENTARY SPOKEN AND WRITTEN FRENCL, by Richard 
Strawn 

INFO: Vous commences Je continue jusqu*a Paris 

QUESTION: --Je Fais qnoi? --Que fais-tu? 

The questions are questioning 

(a) the predicate (Choose one) 

(b) the direct object 

The construction of the question £matches / does not matchj fhe 
construction of the information. 

The predicate of the information [has / does not have] a direct 
object. 

( 

|The predicate of the question [has / does not have] a direct object. 
Ail of which makes it appear that the verb faire [f r] I 

• •• is used as a ’’neutral” verb of activity to, ’question 

(c) other activities (other predicates) 

(d) only yerbs With direct objects. 

(e) only verbs without direct objects. 

. . . must itself have 

(f) a .direct objects if' the predicate it questions does. 

(g) a .direct object, regardless of what it questions. 

(h) np object if the predicate it questions does not. 



(a) 

does not match 
does not have 
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Sample page from ANALYTIC GEOMETRY: THE LINE, by 
Thomas Davis 



225 



C 

sf e 



the y-coordinate of each 
is zero 



5. If we give the three x-intercepts ii 
figure (3) coordinates A(-2,0), B( 1 , 0 ), 
and C(3 ‘1/2,0), what do the three pairs 
of coordinates have in common? 



6 . Thus, to find the x-intercepts of 
the curve whose equation is y=x^ - 4 , 
we set 
solve for x. 



^in the equation and 



6. 

y s 0 


7. Find the x-intercepts of the 
equation 

4x2 + 9y2 = 36 

i 


i 

7. The x-intercepts are 


8. We saw above that in order to find 


the points (-3,0) and 


the x-intercepts of an equation we 


(3,0). If you had dif- 




ficulty, go on to the 




next frame. If not skip 


- - - - . . --- __ __ 


to frame 13. 









8. set y s 0 and solve 
for X 



9. Thus, setting y = 0 in 
4x2 + 9y2 = 38 

we . get 



9. 4x2 s 36 



10« And solving for x we get 



10. X s * 3 
(since x 2 s 9 ) 
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(Cent.) sample page from ANALYTIC GEOMETRY: THE LIME^ 
by Thornes Davis 



226 



>!• 3 and »3 



0 



A 



m 






12. (*-3t0) and 3,0) 



/) 









Si \ 

% \ ^ 



”4 V* 

A'. 






\S\ 

ft * ^ 




14. a point where the 
urve intercects the 
-axis. 



11. Thus the x-coordinates of the x-inter- 
cept are and . And the y-coordin' 
ates are . 



12. Thus the x-intereepts are the points 
whose coordinates are and 



13. In the figure below, label the 
intercepts A and B. 



14. 



A Y-INTERCEPT of a curve is defined to be 
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Relationship to Amount of Time Required to Prepare Programmed 
j Material to Student Time Working on Ppograw 



I An attempt was made to estimate the amount of time the pro- 

gramers worked on their programs compared with the amount t ime 
students worked on them. No hard figures are reported here; mere- 
ly estimates* The reported estimates of time are given in Appen- 
dix C-1. 



The average estimated time spent on the preparation of pro- 
gramed materials was 24 weeks. The average estimated time students 
worked on the programs was 10 hours. The ratio is roughly I'OO hours 
of programers time to one hour of student time. 

Dean*s Evaluation of the Project 

Deans of each college were asked to meet with faculty members 
who had had some contact with the Programed Instruction Project 
during the spring of 1966. They used these meetings to discuss the 
I following questions and to draw up an evaluation of the projects I 

I 1. What have been the strengths and weaknesses of the Pro- 

gramed Instruction Project? 



^ 2 . 
I 

3. 

V 



4. 



What experiences have participants had with using programeji 
instructional materials in your classroQins? (Either 6LCA- 
produced or commercial programed mSteri^ls.) What is 
your evaluation of the effectiveness of fhese materials? 



What has been the experience of participants with pre - 
paring programed instructional materials? 
evaluation of the experience? 



What is your 




What do you think is the future of preparing and using 
programed instructional materials in this campus? That lil 
what place has the preparation and use of programed mator-i 
ials in the future development of our instructional prograiji 



5. What areas of the curriculum and instructional program havi 
been opened for further exploration and development by the! 
Programed Instruction Project? That is, where should we ) 
go from here? j 

I 

Seven deans reported on their meetings. A brief resume of ^ 
the reports including suggestions and insights of the deans follows! 

/Strengths and__weaknesses of the proiect jof ,thfe seven reports 
returned by the deans five mentioned as a major strength that the 







li 

prepering of programs was a good pedagogical experience, that teach- 
' ing had improved as a result and that faculty members were better 
j equipped to organize and evaluate all methods of their own teaching! 

Four of the'deans mentioned that the broad nature of the pro- 
p* gram was a major strength-* several kinds of programs, including 
n' ^^product ion of sli des and many ’disciplines werd included. Programers 
! were encouraged think and work in large educational ideas rather 
I than in restricting themsel jS to "conventional programed materials' 

I These also mentioned the value to the campus as a whole of the idea 
of programing and use of a few completed programs. 



Several deans mentioned the high quality of the training that 
programers received. 

Weaknesses of the project' Included mention by most deans that 
I preparing programs was extremely time-consuming, although it "was 
not tedious". Several suggested that partial of full released time 
1 for participants would have improved both the production and the 
j finished programs. Four deans felt that the evaluations of commer- 
cial programs were not adequate and that the follow up was poor. 

1 Three of the reports referred some danger in the humanities "Of 
I arriving at a sense of absurdity through a programed technique". 

I Some areas of literature, for example, do not have an Innate logic 
land are not "susceptible to the rigidity required." 

I Use and effectiveness of programed materials . There was general 
I agreement that programed materials did achieve their goal, that I 

I studettits did learn the material and instructional ti^me was saved. 

A few faculty persons mentioned that not all students learn well 
I from this kind of material, although whether the specific program 
or the use of programs generally was at fault was not clear. The 
novelty effect proved valuable, and most participants felt the pro- j 
I gram was very good for part of a course. One report stated £ Pro- 
I grams were used in music, logic, physical science, short story, 
economics, psychology, and mathematics.... Our results were mixed 
to very good. Another report reads: the usual comment concerning 
this was "excellent"--it was a discussion of nothing but praise for 
the involvement as participants. Another report stated: In most 
cases ... teachers felt it assisted in focusing student attention upon 
necessity for careful definition and clear distinct ions ... seemed to 
lead to more precision in discussions and a better command of term- I 
inology at an early stage ... several , however, felt that results 1 
achieved may be more superficial than penetrating and tended to I 
show little effect in final essay examinations. | 

Preparing programed materials . All of the deans reported that pro- ! 
gramers found preparing programs to be very time-consuming, but 1 
I rewarding, with insights into the teaching-learning process as a | 
I bonus. One report said: Programed material helps to break "mental 
blocks" and forces students to dig into material they would ordinar- 
ily skim over. • .on the other hand pressure for consistent and paced ! 



I 



j use of programed materials was necessary. 

^^ture of preparing and using programed materials . There was 
cautious enthusi«"sm for the future of both use and preparation of 
programed materials, with "bright” and "promising” the words most 
used. Most deans felt that short programs would be written, more 
film loops would be prepared, more faculty members would see value 
in programed materials. One college is seeking a person to be a 
Director of Communication who will be an expert both in use and 
j production of all kinds of instructional materials who will help 
faculty locate and use programed materials. Several deans men- 
I tloned a need for teams of writers and for subsidy to encourage 
programers to prepare materials. There~may^E>e“~a“W\nrng away fr 
.lectures to more and more self-paced learning, with consultations 
with a teacher at a time of crisis more possible than_formerlv. 



production of better textbooks and of improved understanding of in- 
I structors of the effectiveness of their presentations. 

I Areas for further exploration and development . There was general 
I agreement in the reports that more research and development in the 
.teaching-learning process was indicated, that more programs should 
be produced and more faculty persons involved in at least practice 
iof preparing programed materials, and that for any successful carry 
ling out of such activity funds would have to be found to release 
i faculty members for such activity. 



Other Comments . During the discussions in the vay^ious colleges 
^several pertinent points were made. One group suggested further • 
^cooper ati on- among HLCA colleges including a resotlirces center. 
'jAnother group felt that many college teachers see* themselves as 
professional scholars, not as educators. Several mentioned the 
dl^iar understanding of relationship between teaching and 
learning. .. "development of the theory of learning should be the 
basis for teaching evaluation and improvement.” 



l £valuation of Team vs» Indiv idua l Preparation of Programs 

A second important objective of the extended contract of the 
|Project was to compare two methods of preparing programed materials 
andividual and team preparation on subsequent instructional pro- 

I It has been extraordinarily difficult to implement the team 

|Concept in the process of preparing programed materials. The major 
problem is that of communication, a second, almost as Important is 
that of scheduling. 

1 Ideally, the proposal for preparing programed material should 
|Come-out of a team or group. That rarely occured in the Project. 
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At the time that a proposal to prepare programed material was pre- 
sented to the Director, it should have geen evaluated by two panels: 

1) a panel of experts on programing who would evaluate the feasibility 
of programing the material, and 2) a panel of content experts who 
would evaluate the value of programing the material for the disci- ' 
jpline. The Project relied only on the former panel to make the 
jseleotion of programers for the summers of 1964 and 1965. 

A panel of content experts was not used in selecting t he p ro.-, 
gramers because of physical and financial problems‘^F^e"t'tTng toge- 
ther so many persons-one panel for each area of subject mattor-and 
'because of anticipated problems of getting satisfactory evalu.ations 
from them if they worked individually. Obtaining evaluations from 
'one panel was difficult enough to discourage further attempts to get 
other panels. 

i A second ideal point for interaction between programer and 

^content expert would have been after the time that the programers 
iwere selected and notified of their having been chosen to prepare 
programed material and prior to the time of the first summer- train- 
ing workshop. During this period, however, the college schedule 
worked against the possibility of team action. April, May and June 
,are extremely busy months in college. Although we asked the selectodl 
'programers to prepare an outline of what they expected to program [ 
during the summer before they came to the first workshop, we harbored 
many doubts about their willingness or ability to work even that far. 
ahead* Besides our agreement was for them to write their program 
during the summer. Some of them had only very sketchy ideas of what 
they were going to do by the time of the first workshop. 

. 

During the first training workshop, the content editors could 
'conceivably have worked with the programers. But the latter were 
'completely preoccupied with the process of programing. The same was 
true during the period between the first and second workshops. By 
Ithe time of the second workshop the programers were still highly in- 
volved in the problems of programing. During the second workshop 
they conducted the first developmental test of their own two students 
Testing the program in this manner was in some cases rather distur- 
bing and sometimes pointed to the need for extensive reorganization 
and revision of the program. Content editing seemed to be rather 
inappropriate at this time as well* Besides, by the middle of July, 
it became unusually difficult to locate college personnel unless the) 
have previously committed themselves to work on the project. 

Finally, almost in the middle of summer, the programers began 
t^ be^hla..Jt^^ about using content experts to review their pro- 

grams. At that time the director of the project wrote to at least 
two persons for every programer inquiring into their availability as 
consultant to review the program for a given programer. In most 
cases the programer availed himself of the opportunity to get advice 
from such content experts. 
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Howeveri by the middle of the summer the programs had crystal- 
lized to such an extent that the content editors were not likely 
to be able to introduce a | change in the content of the program as 
they might have been had they been used earlier in the process of 
programing. 

Study of the Broader Effects of Programing 

Objective 6 of tha original contract and objective 2 of the 
extended contract both call for a study of the broader effects of 
programing. After the first summer of programing » the Director 
.sjsnt out a questionnaire to ascertain what effects the experience 
had on the programers concept of teaching- After the second summer 
of programing a more sophisticated study was designed to assess 
mor« precisely the extent and nature of the impact. The results 
of these studies are reported in Part III of the report. 

Summary of Project Activities 

All 6LCA colleges participated in the three-year Programed 
Instruction Project and were kept in touch with the various activi- 
ties of the Project through the Director who communicated with a 
Liaison person on each campus. The Liaison Committee met a total 
of five times. The Director described the Project to the faculties! 
of all the colleges and made numerous contacts with faculty members y 
on the various campuses for a variety of purposes related to the 
project. Two major all-Association conferences were held. The 
first was a series of winter work conferences held early in the 
life o f the Project to introduce the colleges to Programed Instruc- 
Tion', the second was a Final Reporting Conferencf in which the 
results of the Project were disseminated to all the colleges. 

RESULTS 

Thirty-six programs were produced in the two summers of the 
project. They were field tested in 6LCA and other colleges. The 
programs are all relatively short, topical programs. Each college 
has received a sample of all the programs that were submitted 
to the Director *s office. Eight programs are currently being 
commercially published and probably several more will be published 
in the future. 

Five 6LCA produced programs in the fields of science, social 
sciences, and humanities were rigorously and systematically eval- 
uated under a variety of conditions in 8 GLCA colleges, by 26 facu%^ 
members involving approximately 1200 students. The evaluation 
attempted to ascertain whether programs teach as well as other con- 
ventional instructional methods, whether they are best used 'before 
or after classroom discussion of the given material, whether they 
are used most effectively with lecture or with question and answer 
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discussion and the effects of motivation on students* learning by 
means of programs# Characteristics of students evaluations and 
faculty evaluations were also obtained. In addition a dozen com- 
mercial programs were evaluated. 

The impact of preparing programed materials on the programers 
concept of teaching was investigated. 

Forty-two programers were supported by the Project and were 
intensively trained in the principles and procedures of programing. 
Six editors received further intensive training to consult with the 
programers# An estimated 200 (about 15% of the total GI«CA f*aculties 
were directly involved in the project through conferences • preparing 
programs, field testing programs, evaluating them, and through 
contacts with the Director. All faculty members were alerted in 
varying degrees to new media of instruction through the initial 
round of faculty meetings, and two rounds of visits to display' pro- 
grams. The results of the project were reported out at a Final 
Reporting Conferences in Columbus, Ohio on April 29, 1966. Eight 
institutions and agencies other than GLCA colleges were involved in 
the Final Reporting Conference. 




A CONCLUSIONS 



Professors in GLCA colleges can prepare high quality programs. 
Preparing quality programs is a very time consuming process with 
respect to the amount of student time in using programs. 

The deans of the colleges generally evaluated the project 
favorably. 



GLCA faculties demonstrated the value of having an association 
of colleges prepare and evaluate programed materials, the feasbility 
of a cooperative project demanding a high level of coordination and 
cooperation, and the possibility of disseminating information on 
instructional matters. The Project focused the growing interest of 
the teachers on the process as well as the content of teaching, on 
pedagogy as well as scholarship. It can be concluded that GLCA 
faculties and probably faculties of other associations of colleges 
can successfully undertake projects demanding a high level of 
cooperation. 

Let us review the objectives of the Project at this point to 
ascertain in a preliminary fashion the extent to which they were 
attained. The Project operated under two sets of objectives, the 
original ones and those that were formulated for the extension of 
the Project. They are given in the Introduction to this report. 

I In the three years of its life, the Project accomplished the 

‘ following: 



1 . 

2 . 

3. 

4. 

5. 



Reviewed and evaluated commercial programs (original ob- 
jective 3), 

Developed three dozen programs (original objective 4), 

Intensively evaluated five 6LCA**produced programs under 
a variety of teaching conditions (extended objective 1). 

Described the broader-ef facts of programing on instruct 
tlonal processes of programers (original objectives 5 and 
6« extended objective 2)« 

m 

Disseminated results of the Project as far as possible 
both within GLCA Colleges and in other liberal arts 
colleges (extended objective 4), i 



3 



Detailed description of the attainment of the above objective 
is given in the remaining chapters of the report and summarized in 
both the Introduction to the report and the final chapter* 

The Project experienced the most persistent difficulty with 
respect to the attainment of the original objectives 1 and 2 and 
extended objective 3* The first two objectives dealt with the de- 
termination of areas within the curriculum common to the same de- 
partments in various colleges which lend themselves to programed 
self-instruction and the determination of specific behavioral ob- 
jectives to be achieved In those areas of the cuis‘riculum* The thirli 
had to do with team production of programs. Special attention need 
to be paid as to why the Project was less successful in attaining 
these objectives than the other objectives, 

As will be seen in later chapters, the evaluation of 6LCA- 
produced programs did demonstrate that programs dealing with subjecr, 
matter within the three major divisions of the liberal arts 
curriculum all taught effectively. This result indicates tenta- 
tively at least that there is nothing inherent in any of the major 
areas of the curriculum that would prevent programed instruction 
from being developed and used effectively. 

The first two original objectives go further, however, than 
simple successful use of the programs in various curricular areas. 
They suggest that content to be programed originates from within 
the curriculum , that Instructors can agree on areas of the curri- 
culum that lend themselves to programing, and that behavioral ob- 
jectives for such areas of the curriculum can be formulated and 
consensus reached upon them. Underlying these two objectives are 
the following assumptions about GLCA Colleges and the Project: 1) 
That there' was a Sufficiently high level of cooperation attainable 
to make such .agreement on* the curriculum and objectives possible. 
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2) That there was a suf f icierstly high level of pedagogical sophist — 
cation and interest to make possible the determination of behavioral 



objectives and areas within the curriculum that were programable ^ 
3) That the Project had the resources and time to bring about such 
agreement and consensus. 



As indicated in the previous chapters, the above assumptions 
proved to be largely untenable. The visits of the Director to the 
6LCA campuses and the Winter Work Conferences were not sufficient 
to bring about the meetings of minds that was required by the two 
objective©. E\»en the second round of programing failed to achieve 
the extended objective number 3 of program production by team in- 
teraction* As a result, the Project resorted to the individual 
approach, encouraging individual instructors to prepare programed 
instructional materials primarily for their own use and secondarily 
for use by other interested persons. 



Why was it so difficult to obtain group consensus on what 
needed to bo programed and cooperation on production of programs? 
Several answers suggest themselves: 



1 . 



Teaching is a highly individualistic matter in GLCA 
Colleges and obtaining consensus and cooperation runs 
rounter to long-established individualistic pedagogical 
traditions . 



Most |.nstructors were relatively unacqijainted with pro- 
gramed instruction; some were even antagonistic. One of 
the major tasks of the Project centered around not only 
disseminating the products of the Project, but also pro- 
viding fundamental basic information. .Planning rather 
sophi sticated cooperative methods of developing and 
utilizing programed instruction was beyond the competency 
and interest of most faculty members. 



3. 



A msjor question needed to be answered before wide-spread 
interest in programed instruction could be assumed: Are 

programs effective pedagogical devices at all at. the 
college level? Until that question was answered - which 
was done affirmatively by the Project - many instructors 
could not be interested in preparing or evaluating pro- 
gramed instructional materials. 



4. 



Programed instruction has less of a relationship to the 
content of the curriculum than it has to the methods of 



teaching . The curriculum as such failed to generate 
programed instruction. Rather the Project attracted in- 
dividual teachers to prepare programs who were interested 
in experimenting, who were experiencing difficulty in 
communicating to students w]|io were intrigued with the 
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possibility of improving the exposition of ideas • who were facing 
problems in presenting certain concepts, who were concerned with 
improving instruction in their own classroom. The source of moti- 
vation then to produce programed instructional materials seemed to 
be more personal than curricular or institutional. 



The failure of the Project to attain objectives number 3 of 
the extension of the Project is related to the failure to complete!^ 
attain objectives 1 and 2 of the original contract. Programers, 
editors and content editors were practically impossible to bring 
together in a team arrangement because of simple scheduling prob- 
lems >,f bringing people together in the summer time of year .when 
most of them were intent on vacationing or pursuing their own per- 
sonal interests. It still remains to be seen, however, whether a 
group of programers working in concert might not reinforce learnings 
and thereby magnify the effects of programing upon instructional 
processes to a greater extent than programers working individual- 
istically . 



This is not to say that the first two original and the third 
extended objectives are unattainable. It does suggest, however, 
that it may be necessary to have experienced programers rather than 
novices in programing address themselves to the areas of curriculum 
to be programed and to the formulation of behavi^pal objectives. 
With their training and experience in producing programs as back- 
ground they would possibly be able to do effectively what relative- 
ly inexperienced programers found difficult, if i|pt impossible to 
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Chapter 4 



REVIEW OF RESEARCH IN PROGRAMED INSTRUCTION 

AT THE 

COLLEGE LEVEL 




By M. Daniel Smith 



There ^e two general categories 
struction; stressing experimental 

experimental design. The former empha 
a sequence of learning tasks, the latt 
which differ in one of a few variables 
tion s with sophi sticated statistical d 
maXinniumber of subjects, the latter 
statistical significance. . Some recent 
possibility of a compromise (14). Thi 
of research of both kinds. 




of research in ^/programed in- 
analysis, anti/ ttreft stressing 
sizes careful development of 
er the comparison of* treatment 
(or multivariate investiga- 
esign). The former involves 
large numbers for obtaining 
developments point to the 
s chapter represents a review 



Research involving experimental analysis; 



? 

1 



An early attempt at programing was made by Socrates, and re- j 
ported by Plato in the Meno . It involved mathematics and was dis- 
jcussed and demonstrated before a collegiate audience, although the t 
student was’ reportedly a slave boy. It involved rather repetitive 
^responses and was overcued; there is some evidenpe that the author 
jdid not mean it as a serious example of pedagogy. Broad scale 
programing was §lso characteristic of the efforts of the Sophists 
and later the Jesuits; their efforts were develoged with practical, 
specifiable end products ^in mind, and certainly represent an early 
form of behavioral analysis and sequencifig of lefl'ning experiences. 



Skipping some hundreds of years, we note next the work of Pres8<$ 
which involved immediate feed-back from tests; this was conducted 
with attention to experimental design as well. 



The earliest thoroughgoing development of a program was carried 
Holland and B. F. Skinner; they used parts of Science 
And Huma'n" Behavior (by the latter) and expanded them into the well 
‘known program Analysis of Behavior . It was first presented via 
^teaching machines of the disk type. The program went through sev- 
!eral extensive revisions, each based on feedback from large numbers 
'of Harvard sophomores. It was noted that increases of length due to 
addition of frames resulted in decreased learning time and improved 
jresults. Students reacted- negatively to the confining nature of the 
pxperience, both physically and cognitively speaking, but since the 
istudents did much better on final examinations than they had before, 
jthe authors concluded (evidently) that they were justified in over- 
looking the kind of complaint with which less favorably endowed 
Institutions wrestle so timorously. Some experimentation was done 
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with various response requirements; it wasfshown, for instance, 
that the choice of response requirement in a given sentence or 
paragraph is important and determines how much and what the student 
learns. Some work was done with retention, indicating that it was 
not a case of "easy come easy go"; however, the absence of good 
comparative data from "conventional*’ treatments hampered efforts 
in this direction. Generally it became obvious at this early date 
that there was no good way to compare "programed instruction" with 
"conventional teaching", since tl»ore were so many different facets 
of conventional teaching and since there were so many uncontrollabl 
variables which would be involved. Outside of this, very Tittle wa 
done with college students by the Harvard psychologists. The 
author used short programs in physics and mathematics, with some 
success, but also with the negative student reaction attendant on 
experiments involving individual learning styles. The main thrust 
of the research on this pro jec^ turned toward younger children. 



Research in teaching Russian conducted 
ham College is one logical successor to the 






by Keith Myers at Earl- 

v.w-.*v.e,v* W..W uiic analysis involved in 

Analysis of Behavior ; another is the program in statistical infer- 
ence by Celeste McCullough and Loche Van Atta at Oberlinj one 
should also mention the Genetics program by Edward Kormondy, in- 
volved in research design experimentation reported elsewhere. 

The work in Russian by Myers was pioneering in that it involve 
the design of special dual tape-deck audio-lingual capabilities for 
the express purpose of carrying out the needs of the program (un:^ 
fortunately most machine carry out the dreams of ’The “designer 
rather than any particular tutorial requirements); This machine 
design also involved some forward-looking prototypes of language 
laboratory equipment by John Gilpin, an associat^'^on the Earlham 
project. Dr. Myers conducted one experimental class in beginning 
Russian entirely through machine and text, with no contact with 
students except through tapes recorded by them and correction tapes 
made by him. Students questioned four years later reviewed the 
experience as helpful; some went on to advanced Russian after less 
than a term. 

A program in Spanish developed by Rand Morton at the Universitjr 
of Michigan stressed discrimination learning in acquiring Spanish 
pronounciation and vocabulary. 

The work in statistics by McCullough and Van Atta at Oberlin 
was more conventional in foi^mat and subject matter; it too received 
extensive trial and revision, and stands today as one of the better 
examples of the sequencing of learning experiences in topics 
traditionally looked upon as college level. 

I 

j The work of William Stephenson, 0. T. Benfey, Hugh Barbour, 

iHoward Alexander, and Leonard Holvlk on the Earlham project has 
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been reported in detail in a final report of ’^hat pro 3 eot. GeneralH 
ly it was found that programed materials were well 
students, were effective in communicating concepts which had pre- 
viously caused difficulty in their respective courses, but ® , 

great deal of time and effort and persistence to prepare. 
ham project also noted that programs from 

exceptions, were not as well received: either they did no _ 
relevant material, or the material was not collep level in 
Ondef' the^Trei^ure of a three-term system 

even more difficult to get students to experiment with this often 
rather slow approach to learning. It was 

did the student a disservice by requiring him ^o^artt at .a pace 
which was less than that to which he was equal and accustom.ed. 



Since the production of these early programs, a great many avi| 
become available through commercial publishers, often demonstrating 
many of the unfortunate characteristics of early programs as well 
as a lack of understanding of subject 

concerned with traditions of excellence in tea^ifTg, competence in 
subject matter, and concern for the student, have ^®®*^ ° . j 

premature dismissal of a potentially excellent approach to learning, 
one which can demand more of the student and give him greater re- 
turns than is characteristic of his experiences in college today. 



Research Involving Experimental Design. 



The earliest study of programing at the college level employ- 
ing experimental design also employed analysis to the extent that 
it emphasized qareful development of the programed materials before 
using them to compare programing with other approaches to teaching. 

E. J. Kormondy and E. L. Van Atta used a progralii involving the 
field of genetics to make comparisons between ^fveral more or less 
conventional teaching approaches, and furthermd|*e did ™°^® 

than one campus. In a sense, this study, utilising f ® 

Of students from Oberlin and from Earlham, foreshadowed the present 

project. 



The program was described as follows (9)^^ 



ft 



The subject matter of the biology program comprised a review 
of mitosis and an introduction to meiosis and basic genetics. 
This section of the course contains basic information suitable 
for programing and complex and technical aspects fundamenta 
to the mastery of a significant portion of the content of the 
course. The programed materials were first given during the 
fall semester of 1959-60 to 184 Oberlin College students en- 
rolled in an introductory course in zoology, and in collabo- 
ration with Dr. William K. Stephenson of the Department of 
Biology of Earlham College, to 161 beginning biology students 



13 



See Appendix D-1 for Bibliographic references. 
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at Earlham College. A revised version.. was given during the 

fall semester of 1960-61 to 213 Oberlin College students.'* 

The experimenters cast the program as a programed text, and 
also prepared an "experimental textbook; which contained the same 
material as the programed" textbook; for the most part being an 
identical word-for-word reproduction of the programed text but 
arranged in paragraph form and having no blanks to be filled in or 
problems to be solved except that at the end of each chapter." 

The experimental groups did not attend the three lectures presented 
during the experimental period: Experimental group l iused the pro- 
gramed text- Group 2 used the same materials and also a booklet of 
91 review items arranged into four review lessons, and Group 3 used 
the experimental textbook exclusively. Control Group 4 attended 
three lectures and were strcngly encouraged to study their textboolc 
( General Zoology by Villee Walker, and Smith) an additional six 
hours; control group 5 attended the three lectures, studied their 
textbook (different textbook at Earlham), but were not especially 
directed to study in any particular way or for any particular time . 
Control group 6 did not attend lectures but were assigned to inde- 
pendent study and referred to the same pages of their textbook as 
Group 5. 



In discussing their results. 



Kormondy and Van Atta observed: 



"Oberlin, 1959: The three groups using experimental materials 
(groups 1, 2 and 3) performed equally well and sl|gnlf icantly better 
than group 5 which followed a conventional learning procedure. In 
addition the thi^ee experimental groups performed ‘better than group 
4 which used a conventional textbook for independent study: in I 
each of these cases the differences between the liifans approaches 
significance. 



Earlham: The three groups-using experimental materials (groupii 
1, 2 and 3) appear to have done no worse and no better than group 
5, which followed a conventional learning procedure, and signifi- 
cantly better than group 6 which used a conventional textbook for 
independent study. 



Group 3 showed less improvement than either of the other two. 
but... group 3 are computed from a higher protest performance, than 
either of the other two groups. 




Oberlin, 1960: The three groups using experimental materials 
performed as well and showed greater gain scores than groups 5a an 
5b, both of which *followed a conventional learning procedure*. • 

Student reaction* • .was sampled in the 1960 Oberlin experiment 
the majority was favorable disposed toward the materials and pro- 
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sdures used* ••(In each experimental group) 




Kormondy and Van Atta concluded that/students appear to learn 
more effectively and efficiently in ‘independent study with material 
designed for independent study than with conventional textbooks; 
they observed that the groups using programs with and without review 
land the group using the experimental text (based on the program . 

.without response requirements and with conventional paragraph rormatl 
.did equally well* This was an early indication of the relative 
power of the various factors involved in programing t primarily the 
response-requirement factor and the organizing-sequencing factor* 
.Here it seems that the organization and sequencing of the materials 
was more important than the requiring of responses of „ 

It should be kept in mind, however, that the "experimental textbook 
was the result of the programing process; it might be well to con- 
isider it an advanced type of program, which, relies primarily on the 
Iclarity and organization which results from the trial-and-revision 
process characteristic of program development* 



* About the same time as the Kormondy study there was general 

preoccupation with the problem of programed text versus teaching 
machine* Goldstein and GotkinO) laid this to rest fairly well in 
a review of research, some of which used college students among its 
‘subjects* None of their studies reported significant differences 
•in mastery of subject material between machine and programed text 
Ipresentation of the same programed sequence* The texts saved time, 

jhowever* 



Vi 



Also about that time Roe (11) compared branching methods for -- 
.programed learning, using freshman in engineering at uCLA, and foundi 
that "careful sequencing of items has a signifan^ effect on student 
performance, at least for programs of some length and complexity* 

While there was a significant difference in learning time (but not 
in text scor'g's) favoring the forward over the backward branching 
■procedures, there was no significant difference in either learning 
time or test scores between the forward and the linear methods* 
■~l'thereforeT^h^^ sTinple branching procedures tested here do not, by 
;and large, seem to be more effective than a linear procedure**.. 
iThis was again one of many studies which cnlminat^^d^^n no signifi^ 
^differences between the treatments studied ‘also typified the 

iadaptation of conventional research approaches to programing research, 
in that the materials are described as consisting of "six introduc- I 
itory and instruction items, 93 items covering seven concepts of 
lelementary probability (relationship between information, degree of 
certainty, and probability measure; sum of probabilities***), and 
'l4 criterion test items (two for each of the concepts). The pro- 
gramed material- was administered first to a few individual students 
revised, tried in a pilot study with 55 students in a similar fresh- 
man laboratory during the spring of 1961, revised again, tried on 
six individual students, and revised for the last time prior to the 
September 1961 experiment." The length of the program, and the 
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I large-group nature of the trials which preclude a close analysis 
j of the effect of the type of sequence and nature of task on the 
j individual, place in some doubt the effectiveness of the program 
and its appropriateness as an example of what "programing” can do, 

I Della-Piana (2) wrote a report regarding programing as related 

to motivational characteristics of the learner; using what was a 
reasonably well developed program on Rogerian counseling with 
college juniors in a course in psychology for secondary school 
teachers, the experimenter came to a number .of conclusions of interl- 
©st. In discussing an item-analysis of post-program measures, it 
was concluded that "the program was not as effective in getting 
subjects to be able to label new counselor responses and to produce 
specified counselor responses as it was in getting responses actual 
ly called for in the program, and that the tests pointed up the 
kinds of behaviors that need more effective programing. It is 
clear that many students are able to produce written counselor - 
responses (of a specified type) even though they were not called 
on to do that in the program, but that more effective production 
of this behavior might follow actually calling for such responses 
in the program " 
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Another conclusion was that data regarding different response 
modes which were used for different groups indicated no significant! 
differences, and confirmed in part the hypothesis that the im- 
portance of response mode is inversely related to the probability 
of correct responding; i,e,, when the program is easy, differences 
in rjesponse mode are not significant. This conclusion was based 
orig.tnally on work with overt versus covert responding, where 
response mode was important only when the program was "difficult" 
(in some cases, the material was essentially unprogramed,) Della- 
Piana also involved motivation in the study, through use of the 
Osgood semantic differential; results were not particularly infor- 
mative, although it was found that the subject's performance on 
the constructed response version of the program was correlated 
with his evaluation of the treatment, implying the better motivated 
subjects performed better. 
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Green, VIdiss, and Nice explored the use of a programed text 
a medical school course at Dartmouth College, using a course in 
Parasitology enrolling twenty-two students. The original program 
was tested for error rate and edited to insure accuracy after use 
by one class during one year, and was rewritten before the ex- 
perimental use. The experiment was designed to provide two counter 
balanced groups with each student serving as his own control; both 
groups, A and B, received a common core of instructions consisting 
of an introductory lecture, films as they pertained to the subject 
and common laboratory periods. While Group A followed conventional 
projL ftdurea Group B used a programed text, and vice versa; when 
using the program, each group was instructed not to attend lectures 
or use any other reading material other than the program. In the 
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discussloDt the experimenters observed that: 



"The demonstration of the effectiveness of programed instruct- 
ion in the experiment rests primarily on sue examination results. •• 
performance on the examination was significantly superior at the 
.. 01^ level' to that performance on the same examination for which the 
students had prepared in a conventional manner .. .-rem'^performance 
resulting from non-programed study is much lower for the bottom 
five students than it is for the top five... The difference between 
programed and non-programed performance for the top five students 
is not statistically significant. The difference between programed 
and non-programed study for the bottom five students in te.rms of 
examination performance is statistically significant beyond the .05 
level. ... The total study time for non-programed instruction is 
significantly greater than the time spent in programed instruction 
for both experimental groups » separately and for the entire class 
combined. ... The program emphasizes the points which the programer 
considers to be important for the student' know, jrhe j^udeni; is 
therefore saved the effort of reading a* large mass of material* and 
sorting forhlmself those essential points from less important as- 
pects of the material..." 

The experimenters also made an observation concerning the 
"difficulty" of the program: _ 



"The density of the program used in this • experiment was very 
high compared with other programs commercially a*' »ilable for use at 
the college level. Previous experience with t a earlier form of 
this program indicates that, with students of high ability, the 
••programed materials should be more difficult in order to sustain 
interest and to maximize effective learning. This is not to Imply 
that the program should constitute an examination, but rather to 
indicate 'that“““tTie larger step size of a program for students at this 
level is a more effective device." 

This confirms observations made previously in the Ear3.ham 
College Project report, described briefly above. 

A study of the use of programed instruction presented via 
television was conducted at Penn State by Carpenter and GreenhilKl 
One objective of this study was to determine whether there were 
differences in learning programed material over television in com- 
parison with the conventional face-to-face instructional class 
covering the' same subject matter. Thirty college students were 
randomly placed into (1) a televised program or (2) face-to-face 
non-programed Instructional treatment. Each class met for the same 
amount of time, and a pre<^test showed no differences in the groups. 
Both groups learned: no differences in performance between the two 
treatment groups were found on any of the four post-tests which were 
used to measure learning. Students in the face-to-face group re- 
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ported more positive attitudes toward their instructional procedure 
than did the students in the televised treatment* 



On another study • involving English grj?mmar programing, with 
two students using one program simultaneously and discussing diffl- 
cyi.'t p oint s, it was indicated that personality variables takeiTlnto “ 

m. V .9 . m. A ^ _ 
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account Tn pairing the students did not effect subsequent perfor- 
mance significantly* 



Using a program entitled "The Exfosltory Use of the English 
Language" on the 1962 entering freshman class' at Fairleigh • Dickinso i 
(8), these experimenters found that verbal aptitude was measured by 
the SAT-V appeared to be positively related to gains in writing 
ability (as measured by the STEP-Essays) but not to gains in the 
ability to recognize correct English usage (as measured by the 
STEP-Writlng Tests)* They concluded that students did not learn 
the expository use of the English language better through out-of- 
class programed Instruction than through conventional out-of-class 
assignments* They comment that: 



"the results of our study underscore the great need for exten- 
sive research into the linguistic habits of native speakers as well 
as the need for finding appropriate programing techniques for con- 
veying sophisticated, complex content in an accurate and interest- 
ing way*" 



A two-way design of analysis of variance wa^i used by Moore and 
Smith (10) to compare the effects of knowledge of results, know- 
ledge of results plus knowledge of what the corr-^ct response should 
e, knowledge of results plus an extrinsic rewards no knowledge of 
^^esults, and the effects of using a teaching macl^ine as a programed 
ext for two modes of responding (multiple choicij vs* constructed 
esponse) in a programed instructional format on the achivement 
nd attitudes of Introductory psychology st udents . Eq u al n ujube rs 
pf Ss from each of two sections were assigned to each of the groups 
ps met with their regular classes for two one-hour periods each 
Week for 15 weeks, and they met with their respective experimental 
roups for two one hour periods each week for six weeks* 1,152 
rames of the Holland-Skinner program were used, adapted to mul- 
iple choice for that treatment* None of the treatments re- 
ulted in significant differences on the criterion measure, which 
was two achievement tests constructed for the experiment* The 
iuthors felt that the failture to obtain significant differences 
30uld be attributed to lack of sensitivity in criterion instruments 
lifferences in learning strategies adopted by the groups to make 
up for treatment- differences, or perhaps other variables, but that 
it was quite likely that the treatments themselves actually did 
iraake no difference* While it is not explicitly stated in this 
utudy, the assumption has often beeh apparent in other studies that 
cnowledge of results (presentation of the correct answer) in equiva 
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lent to reinforcement in the three-term contingency; it is becoming 
clear that this assumption is unwarranted, and the results of Moore 
,and Smith may be considered in part further confirmation of this. 



In another more recent study, Moore and Smith (14) investigated 
learning set formation in continued discourse via programed materialp, 
:one with a population of 315 college students. The major independent 
variable was the number of programed units experienced by S; the 
Idependent variables were an achievement test score and the error- 
pate for each program studied. For college Ss, achievement and error 
■rate varied as a function of the number of previous programs ex- 
iperienced, independent of order of presentation of the programs. 

{The effects were positive for the first programs experienced, and 
jtended to become negative for a fourth, fifth, or sixth program. 

.The experimenters suggested that the findings might be a function 
|of the interaction of cognitive, motivational, and attitudinal 
ifactors. This study seems related to a problem in student reaction 
{to programed instruction which has puzzled investigators for some 
|time--that where the student, although reacting favorably to the 
programing approach, finds it mere and more difficult to return to 
Ithe program after breaks lasting from hours to days. This has seemed 
[to the author of this review to be related to something akin to 
eactive inhibition; it is as if the student bui ^t u p a negative 
eactlon to responding to a program and as if this reaclTidn^^aiPe 
umulative in nature over a learning session. If); a recent study 
epo"^*ted in Journal of Educational Psychology, Se vere grouped 
{according to demonstrated reactive inhibition (hi vs. low) and were 
compared on posf-test performance after learning ^through programed 
instruction; the high-reactive-inhibition group was significantly 
superior on this criterion. This neither confir||is nor contradicts 
|the previous observation, but it does suggest sti^^ngly that reactiv^ 

f nhibition is involved, that there is such an "alienation” effect, 
nd that this factor in learning from programed instruction needs 
to be explored extensively. 



Moore and Smith conclude further that "the learning set appeared 
to be formed during experience with the first three programs, re- 
gardless of the order in which the programs were presented and re- 
gardless of tho subject matter of the program. Further, experience 
With three additional programs tended to have a slightly negative 
{effect on achievement as represented by gain scores. They hypothe- 

E ize that the college Ss initial responses to the novel situation 
ere strongly positive, but that by the beginning of the fourth week 
College Ss shifted toward the "more of the same" position. This 
Suggest to them that the learning set is largely motivational and 
attitudinal. 

Summary 



This review has not been intended as an exhaustive summary of 
■research in programed instruction at the college level, but rather a 
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review of some studies which collectively give a representative 
view of the state of research. It does not seem to the author 
that things have changed too markedly since 1960» although some o 
the more recent studies have brought up problems which are a bit 
surprising and promise fascinating results in the future. One 
area of research which has not been included here is that involving 
computer based instruction. This area has not as yet developed to 
the point where results differ markedly in nature or import from 
those reported, particularly at the college level. 



In closing, it may be appropriate to observe that the research 
designs used in exploring programed instruction at the college 
level have not in general been sophisticated, nor have they in- 
volved consideration of many of the variables of current interest 
in educational research. There has been little to relate the 
reactiv‘>ns of students (as well as their performance) to creativity, 
anxiety, apd other special factors. There has also been very littl 
in the way of multi-variate investigation of the various factors 
involved 5 such studies as Siegel* s "The Instructional Gestalts A 
Conceptual Framework and Design for Educational Research" should be 
imitated in their use of multi-variate design. (12) Where programs 
are involved, however, additional r e s t r i‘5“fToTrs--'ai!'i^^'h“T^ 



importantly, the programed material should represd|it the outcome of 
an intense behavior analysis involving individual-4$tudents and 
small groups. Qttly in this way can the experimenter achieve an 
approximation to';an optimal path from the initial ptate of the 
learner to the ||nal state, and only in this way ^^n one assert tha 
the research, o|f)pe done, has relevance to other research involving 
programed instruction. There are many variables"* jrithin programing 
whose interactions are not yet understood; as earl^ as the Kor- 
mondy study it became apparent that the organizational — sequential 
variable might be more powerful than the active-response or feed- 









back variables, and that what we recognize as "programs" may be 



only a stage or phase in the development of the highly effective 
instructional sequence of tomorrow. 



' Finally, we should abandon the programed instruction vs. con- 

jventional instruction approach as misleading and futile; the pre- 
ponderance of n.s.d.'s in such research confirm and reconfirm the 
view that the number of variables involved and lack of control over 
them make*) worthwhile research impossible here. Let us accept the 
fact that programed instruction does teach and face the related one 
that in "conventional" instruction we actually teach much less than 
we are willing to admit, much less reveal through controlled studie 
The reactions of students are not a final criterion for use or non- 
use either, but testimony concerning the aversive effects of some 
programs cannot be swept under the r»*g; as one of the studies re- 
ported above concluded, and as was observed as far back as the repo 
ion the Earlham project, we have not as yet matched the sophisticate< 
learning procee^ee of college students with : 2 ppropriate sequences 
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!of tasks. This does not mean, however, that we cannot. 
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Chapter 5 

EVALUATION OF COMMERCIAL PROGRAMS 
INTRODUCTION 
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One of the original objectives of the project was to review 
existing programs in order to identify those which could be used in 
various departments and to arrange for detailed evaluations and 
field testing of these programs. 

On several occasions the Project Director provided the oppor- 
tunity for faculty members of GLCA colleges to examine, use and 
evaluate commercial programs. The first of the opportunities was 
•offered in January of 1964 when the Project Director sent out a 
ilist of materials that were e^vailable in his office to the liaison 
jpersons to be distributed to department chairmen. The second oppor<‘ 
jtunity occurred during the Winter Work Conferences in 1964 when 
^commercial programs were displayed at the conferences. The third 
jopportunity occurred in May of 1964 when the Director visited each 
of the colleges and displayed the programs for a whole day on each 
of the campuses. 

The faculty’s response to the commercial programed material 
•could be best characterized by the term indifference with occasion- 
al displays of interest. Although a fairly large .number of faculty 
members were exposed to commercial programs, only a few of them 
showed interest in using the programs and still fewer in systema- 
tically evaluating them. 

The indifference was not wholly unexpected. By no means were 
all the 150 commercial programs that the project accumulated of 
college level quality nor in areas that would appeal to faculty 
’members in GLCA. Some of the faculty members indicated that the 
bnly way they could use the programs that were available in their 
field would be as remedial programs* In addition, the greatest 
jproportion of the programs was in the field of mathematics. The 
jsciences were the next best represented. The modern foreign lan- 
|guages were the third. The rest of the disciplines were representec 
{by only a smattering of programs. In a small number of the disci- • 
Iplines such as history, political science and speech no programed 
materials were available. 




er|c 



Besides not having a sufficient number of programs to evaluate 
in the fashion called for by the contract with the Office of Edu- 
cation, the procedures for evaluating the programs were not agreed 
{upon nor wo'rked out during the spring, summer and fall of 1964. 

>In the third meeting of the Liaison Committee in Hay of 1964, the 
{question was raised about the advisability of testing many programs 
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qualitatively versus testing a few programs quantitatively and in 
depth. It was decided at that meeting to test fewer programs in 
depth. At that meeting it was also agreed upon that the Director 
would visit each of the colleges to display the programs that were 
available for evaluation. It was notf however, decided how these 
programs would be evaluated. That is, the evaluation design was 
not clearly thought through and agreed upon. Subsequent events in- 
dicated that the lack of clarity of an evaluation design was one of 
the major handicaps in obtaining useable results from the evaluation's 
that were conducted on commercial programs. 



During the summer and fall of 1964 the Director contacted 6LCA 
faculty members who were interested or who might become interested 
in evaluating commercial programs and setting up with them the 
procedures for doing so. 



In order to coordinate the evaluation project Dr. Donald. Beane 
of the Department of Education of the College of Wooster was en- . 
gaged by the project to serve as evaluation coordinator for the GLCAl 
in Ohio during' the summer and early fall of 1964. During this time 
he contacted llie evaluators and helped them set their designs as 
carefully as possible. The Pro-j^ct Director coorainated the eval- 
uation of commercial programs in the Indiana and - |t|chigan colleges 
of the 6LCA. 



METHOD 



An attempt was made to set up a common format in which all of 
the commercial programs would be evaluated. The attempt was also 
made to use common instruments for the collection of data wherever 
possible. To this end, each evaluator was given an instructor*s 
worksheet in which the design of the evaluation procedures was 
outlined. Each evaluator was also given an Instructor Program 
Evaluation Form. A questionnaire called ^ Student Reaction Question 



naire was used by all of the evaluators to gather the subjective 
reactions of the students of the programs that were being evaluated. 



(See Appendix B-2) 



The Instructor* s Worksheet contained the identification data of 



the name* of the instructor and department, his institution, the name 
of the program to be tested, the course in which the program was to 
be tested and the number of s judents who would take part in the 
evaluation. 



Next the evaluator in* discussion with either DeHaan or Beane 
decided on what .kind of a hypothesis or evaluation question would 
be studied. The three options were (1) the program as a primary 
teaching device* (2) the program as supplementary, or (3)^^jcompari< 
son of one program to another. Any important assumpt lons^"tiraT*‘we»r' 
made about the nature of the program or the nature of the subjects 
were spelled out. ^ 






Ids 



o 

ERIC 






k '■‘•■Si 












il' 



PS 







The experimental and control groups were then defined in terms 
of the number in each group; the manner of selecting subjects for 
each group either by ordinary college selected sectioning proced- 
ures, random selection, or stratified random selection; and the 
selection variables used in selecting the subjects such as by sex, 
SAT scores, achievement test scores, grade point average, or grade 
level in college. • 

The independent variable, that is, what the instructor would 
do, was then defined as carefully as possible. The instructor was 
ashed to name the dates during which the evaluation would be car- 
I ried out. He was then asked to administer and control the follow- 
j ing variables. 

s (a) Supervision of students, what kinds of questions would b** 

(answered, what kinds of help would be given. 

t 

I 

(b) Motivation and control of competitive factors such as 
•grades and tests* 

I 

(c) Control of unscheduled use of programs to avoid contam- 
jination arising therefrom. 

I 

I (d) Control of time spent on program--free versus restricted 

time . 

.i 

(e) Control of experimenter bias — verbal §nd non verbal 
‘■communication of instructor's attitudes toward the program. 

•i * 

\ (f) Determination of use of program; how t^^^handle differences 

land individual speeds in working through the programs. 

I (g) Use of supplementary materials. 

These variables were considered not only for the experimental 
group but also for the control group where these were used. 

The instruments to be used in the evaluations were also dis- 
cussed and agreed upon* Whenever possible, pre and post tests were 
jused to measure achievement from the program. Time sheets and othe 
[records of utilization were also suggested to get an estimate of 
I the efficiency of the student's use of the programs. Student Pro - 
Igram Evaluatio_n Que s tionnaires , already mentioned, were used to 
jcapture the attitudes of th.e students toward the subject matter of 
Ithe program. The Instructor 's Program Evaluation Form was also 
'used to obtain the Instructor's attitude toward the program. (See 
I Appendix B*i). A roster form for recording and summarizing data 
[such as the grade point averages of the students, their SAT scores, 
land other information was used to get a description of the popula- 



tion. 



-■J 



u 






. I 

',3 






Provision was made for obtaining clerical assistance and for record 
ing the amount of clerical assistance required. Finally, a deadine was ' 
set and agreed upon for completion of the evaluation projects. 



The list of evaluators of commercial programs, their departments, 
colleges and the programs evaluated are given below. 
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Mary Lane Charles 
Department of French 
Earlham College 

Ellane Burroughs, Chicago: Encyclopaedia 

Britannica Press. 



Dean Dillery 
Department of Biology 
Albion College 

\Cells: Their S tructure and Function by Marta Zaborska and 
Frances Unger, Meade, Chiddgo: Coronet Instructional Films, 1963 



Larry Hackstaff 
Department of Philosophy 
Wabash College 



Class Logic by Biyth and Jacobson, Mew York: Harcourt, Brace 
World, Basic Logic and Auto^-text . — " 



Edward Kormondy 

Department of Biology ^ 

Oberllt) College 

Introduction t o Genetics by Edward. Kormojidy , Mew York: McQi*aw- 
H ill, 1964 ‘ ' 



-I-jia— Land&n- 



Department of Psychology 
Denison University 

Analysis of Behavior Parts I 6 II - James Holland and B. F. 
Skinner, Mew York: McGraw-Hill 



John Reinheimer 
Department of Chemistry 
College of Wooster 

_^ogyame d Supplement for General Chemistry Vol I 6 II, by 6< pdoi 
Borrow, et . al . Hew York: W. A. Benjariiin Inc. 



Fernando Rodriquez 
Department of Romance Languages 
Ohio Wesleyan University 

french Phonetics - by Eliane Burroughs, Chicago: Encyclopaedia 
Britannlea Press. 
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10 . 



11 . 



Ruth Smyth 
Department of Mathematics 
College of Wooster 

Donald Van Llere 
Department of Psychology 
Kalamazoo College 

with 

Miss Barbara Arnold 

Senior Psychology Major , 

^alysis of Behavior by James HollandLB. F. Skinner.. New 
Yoric: McGraw-Hill '* oKxnner, new 

Robert Weiss 
Department of Speech 
DePauw University 

»«" Vork: 

Joseph Wetmore 
Department of Education 
Ohio Wesleyan University 

|Ht|s|||2iMeasu^ - Gorow, Chandler Pubjlshing Co. 
i^amfrand Co^pany?""'^'' ' K^Clelland. Scott 



12 . 



13 . 



14 . 



Dorothy Whitted 
Department of English 
Ohio Wesleyan University 
Effective Writin^r 

Smith and Stapleford, Garden City. N.Y.: Doubleday and Co. 

Francis Williams 
Department of Chemistry 
Antioch College 

^s^m atie Naming of Aliphatic Comooimda - by 0. Thecdor ' 
Benfey, EafTham College, 1964; ^ ^ ^ Jheodor 

Robert Wilson 
Department of Mathematics 
Ohio Wesleyan University 

i - a - l - eril!s*'gorpf‘’‘^‘^'^"^ * ®*"*''y* Teach! Jg 

llw ‘ ^?orkf Sige^^°^ Alfiebra . by 1. Drooyan and Hm. Wooto";; 

^ d^rrni5n"c°rp f ^ ^.?^'^^^ 
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RESULTS 



Summary of Responses on Instructor Attitude Questionnaire 



Each of the professors who conducted an evaluation study of one 
of the commercial programs filled out a questionnaire. Thts question 
were designed to sample instructor attitudes about the value of the 
program as a teaching device. A tabulation was made of the re- 
sponses on the sixteen Instructor questionnaires which were com- 
pleted! and a summary is presented below: 



8 



Question § 13: 



In your opinion does the program actually teach 
what it claims to teach to the students for whom 
it is presumably designed? 



10 Yes 



jO No ^ Partly 



^ We see that a large majority felt the programs accomplished 
their objective of teaching subject matter to students. However! 
the instructors did find weaknesses in the programs as the following 
series of questions indicate. 



Question # 10: Are there errors in the content? 






0 Yes 10 No 



Question # 11: If you found errors in the content, were they major 

or minor? . • J 



0 Major 



4 Minor 



2 Both 



Question # 12: 



Are there any technical program errors, e.g. poor 
directions, points of confusion, etc.? 



i 



* 2 Yes ^ No 

■j ^ An interesting observation from Questions § 10 and 12 is 
that the wins tructors found more fault with the content of the programs 
than with the manner in vmich the content was presented. 



Question # 14: 



Are the responses that the student is required to 
make token or real? (That is, do the frames merely 
require mechanical filling in or do they require 
thought and/or problem solving?) 



‘ Mostly token responses' 11 . Mostly re^l . responses 
Spotty,, some of- both 






Question" § 16: Evaluate the pace of the program. 

0 Too fast 4 Tob Slow 3 Uneven 
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£ About 
~ Right 
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Question # 18; Does the student get a sense of direction, wh^re he 

Is going, from the prograa? 



~ ^ ®. times 

« 

Question it 19. Can the student easily review the programed material? 

10 Yes 4 No 2 At times 



The responses Indicate that Instructors were in general favor- 
ably Impressed with the manner In which commercial programs presented 
tne content* ~ 



^ Then came a series of questions concerning the practical aspects 
of programed materials. 



Question # 21; Does the importance of the content covered justify 

the amount of time required of the student? 



ii. ^ No 1 Undertain 

Question # 22; Does the program save the instructor time? 

10 Yes 4 No 2 Uncertain 

Question it 23; Is the program justified in terms pf its cost? 

2 , ff. No 6 Uncertain 



« 1 reservations about the quality of the programs 

evaluated, the instructors indicated they did not. know of better 
programs available. 



Question it 25; Do you know of better or similar programs on com- 
parable material? 



2 Yes 



13 No 



Finally, a concluding question about general attitudes toward 
programed materxals was this; 



Question it 31; In summary, what is your overall reaction at this 

time to using programed material in this course? 



£ Very positive, would like to use similar material 
again. 

£ More positive than negative reaction to it. 

£ Indifferent to using it. 

£ More negative than positive reaction to it. 

£ Very poor, would object to using !♦ again. 



no 
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Metho ds and Results of Individual Evaluation Proieets 

Below Is a desc^?lptlon of what each evaluator did in evaluating 
18 program* how well the students achieved with the program* the 
student s evaluation thereof* and the instructor's evaluation of 
the program. 

1. Professor Mary Lane Charles of the Department of FrencJu. 
Earlham College evaluated Purrough's 'fi^h^clf P ho lie trie's 1;*^ TEMAC oro- 
gram. The program was compared against Professor Charles' own pro- 
cedure for teaching French pronunciation. 






loci Method. In the first week of the semester in the fall of 

1964, Professor Charles gave one group the TEMAC program and the 
second group her own material on French pronunciation. There were 

5 students in the experimental group and 5 students in the control 
group. 

The experimental group was told how to use the program. Pro- 
fessor Charles did not drill the students in French Pronunciation* 
only answering their questions. They were graded on French pro- 
nunciation. Tlje programs were used only in the language laboratory 
were kept. Students were given as much time as they 
needed on the program and were asked to keep recoi|d of their time. 

- W 

There were 5 students in each group. They were selected by 
ordinary college selection procedures. The Instructor, however, 
found out from the students which ones had prevlodsly had contacts 
with French and tried to equate the^^number of them in each group. 

She also tried to balance the group in terms of sex and grade level 
in both groups. 



The main instruments that were used were a pre and post test 
consisting of a standard French paragraph. The comparison of the 
groups was made on the basis of gains on the pre and post tests. 

b. Results 



N ^e test Means Post Test Means Gain Score Means 
Program 5 53.2 

Control 5 59.2 



78.2 25.0 

78.4 19,2 



The test used was a 20 item pronunciation test scored on the ' 
basis on for each sound. This was evaluated by three persons* 

a French student asilistant, a French teacher and the regular in- 
structor* making a total possible score of 120» 
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Both groups showed appreciable gain in scores from pretest to 
post test. No test of significance of defferences in mean gain 
scores was made* 

c. Instructor Evaluation 



i 



The instructor described her attitude toward programed material 
in the following words t "I like the organization of the material. It 
is well presented by the speaker. The field of French pronunciation t 
within the scope of this undertaking^ ic well covered. The steps 
are small enough for the student to understand each one. The matei^ia 
is repeated in different ways to furnlah a review of previously pre- 
sented material. I do not see how it could be improved." 



1 



Her overall evaluation and reaction to the program was very 
Ipositive and she would like to use similar material again. She added 
"For best results a course in beginning French should have supple- 
mentary work in the language lab with a teacher correcting mistakes 
in pronunciation* I do not think that this program alone is adequate 
for teaching French pronunciation to beginner." 



f 



i 2, Professor Dean G. Dillery of the Department of Biology, of 

Albion College, evaluated the program entitled Cells: Their Structure 
; and Function , produced by Coronet. 

a. Method. 



ii 



The program was used to supplement the regular materials in the 
icourso. A total of 120 students were used in the experimental group* 
iThe students were divided into two groups of two laboratory sections 
each. Sixty-four students were in the control group that was made 
up of two laboratory sections. These students were selected by cr- 
idinary college sectioning procedures. 

Two lab sections of the experimental group were told how to use 
;fhe program and sections of the program were assigned for five units 
‘of laboratory work. No mention, however, was madecbf the program ia 
any of the lecture sections* Another two laboratory sections of the 
, experimental group were told that the program was Available in the 
library and were reminded once that it was furnished for their use. 
.Students wore not told how to use the program nor i^hs the program 
'assigned at any time. The two laboratory sections of the control 
(group were not told about the program by any of the staff members 
'.involved in Instructing them, except that the investigator did and 
jthen only when the student came in and asked specifically about it. 

’ The programs were placed on closed reserve in the library and 
;time cards were kept on each program. A record of the use of the pro • 
'gram by every student was Kept, including how long he used it. 

It was assumed that if the students considered the program bene- 
’ficial they would continue to use it through all five sections of the 
Work it covered. It was also assumed that if students considered the 
program beneficial they would tell their friends who had not pre- 
viously been told about it. 
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The variables used to measure the effectiveness of the program 
were the number of students who used the program* the amount of 
time the students used the program as recorded by the librarians* 
their grade point average and their scores on the hourly quiisaes 
that covered the material in the program. 



b. Results 



The number of students using the pregram was compared with the 
final semester grades. 



Group 



Semester 



No. in No. Using Percent: 



Experimental Gp. 1 
(Program Assigned) 



Experimental Gp. 2 



Control Gp. 

(No Mention of 
Program) 



Grade 


Dlstr . 


Group 


-^Program 




A or 


B 


23 


8 


35% 


C 




18 


9 


50% • 


D or 


E 


11 


0 

! 


0% 


A or 


B 


30 


6 


20% 


C 




14 


3 » 


22% 


D or 


E 


13 


2 


13% 


A or 


B 


24 


0 


0% 


C 




34 


2 


6% 


D or 


E 


3 


0 


0% 




© 



A greater percentage of the C students in all three groups took 
advantage of the program. The D and E students for which the pro- 
gram a-ras intended made little or no use of the program. 



One noteworthy observation is that students of average or better 
in achievement were more likely to read material if it was assigned 
explicitly rather than merely mentioned. Requiring the program to 
be read had' no positive effect on the D and E students. 



Instructor Evaluation 



The instructor mentioned that he had not used programed materia 
beforse H® his initial reaction to programing as favor- 

able. He said it was "fine for some things and nbt so fine for 
others.” He went on to sajr that the material was aimed at the high 
school level and was much too elementary for college freshmen. 
"Therefore* It <fas of little* if any* value.” He mentioned that the 
program did cover essentially the same material that he desired to 
cover in the course » but did so too superficially* 
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3. Professor Larry Hackstaffi Department of Philosophy t Wabash 
allege, evaluated two commercial programs. 



a. Professor Hackstaff made the following report on his used of 
the programed text, ^asic~liO^*c"Tn?”"Airt6~f exf\ ^ » during the 

.spring semester of I9fe4, v. _ 




; In response to the question of whether he had ever used pro- 

igramed material before, Professor Hackstaff indicated that this was 
his first experience with programed material. He used it as a pri- 
jmary teaching device in place of a textbook. 



J li 




was 
to see 



His initial attitude toward programed instructional material 

He d escribed himself as being "neutral". "I wanted 
the material would work." 



uncertain, 
how 



In using his own words, in giving his reacti on and eval uatJLQJL-tO- 



the programed material, he said the following, " 6a s i c Lo g i c is un- 
satisfactory as a teaching device in logic. It fails to teach the 
students how to make inferences since it presents the material in 
such a mechanical way. This leaves the students to expect an auto- 
matic decision procedure for logic in general — a procedure which 
is not available. The text also contains a number of logical howlers 
which even elementary students were able to detect. My estimate is 
that the author of the text is not a professional logician." 



ti. 









In addition. Professor Hackstaff Indicated there were 

errors in the cont|l|nt of both the major and minor variety. Although 
the program actually teaches what it claims to tea4t|» Professor 
jHackstaff indicate! that most of the material in lEjaSic Logic is of 
ho use for developing a sophisticated course in logic. 



li 



n 
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! He indicated that the importance of the content did not justify 
^the time required of the student to go through the ’program. Neither 
jdld the program save the Instructor time, nor was the program justi- 
fied in terms of its cost. 






When asked whether a similar program was available, or better 
prog??ai(n, Hackstaff indicated that he thought Profess or Morton Seha- 
grin'tj program. Language of Logic , would be a better one. ““ 



ir-a 



When asked to compare the program to both textbook and lecture. 
Professor Hackstaff indicated that he would prefer both the text 



pnd the lecture. "I would not recommend the RCA text to anyone for 
any purpose." 



b. Prof. Hackstaff made the following report on Class Logic , 
k programed text by John Blyth and Jacobsbn, which he used during 
Janua£»y, 1965. 
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The program was used as a primary teaching device in the place 
of a textbook. His general reaction to the program was: “ Class 
Logic was in most ways superior to Basic Logic . For the first fiv« 
or six sections, both students and I were favorably Impressed with 
the book, but by the end of the semester almost all of us were 
of the opinion that the material was being introduced at a pain- 
fully slow and mechanical rate. Good students reported being bored 
by the repetitiveness and poor ones often simply did not do the 
frames. The text covers too little of the subject matter of logic 
and takes too long in doing it." 

In answer to the question, does the importance of the dontent 
covered justify the amoung of time required of the student, Prof. 
Hackstaff answered "no". E’urthermore the program did not save the 
instructor time and probably was not justifiable in terms of the 
' cost for the purposes of Prof. Hackstaff 's course. 



In summary, Hackstaff said the following: "I have yet to be 

j convinced that the programed methods will substitute for a general 
* technical text in the subject of symbolic logic esccept for intro- 
duction of special material. I agree with John B^yth that it may 
be an error^Jrq^jtry; to program an entire logic course. Programed 
; material on aspects" of logic or on material of ap appropriate sort 
|(e.g. symbolism, application, variation on standit*d logic, etc.) 

would seem to me to be desirable. Morton Schag^|p*s program, the 
! Language of Logic , covers a limited and clearly di^iarcated field 
! and is a good start (See Chapter 3 for a description of th^ pro- 
igram.) But my experience with running an entir^ course on the* basis 
of programs plus supplementary material (mimeogyft>hed )* indicated 
i that little is to be gained and something is to lost by structup- 
i ing the course around presently available material." 



4. Professor Edward J. Kormondy, Department of Biology, Oberli 
College, compared his own programed text. Introduction to Genetics , 
McGraw-Hill Company, 1964) and a traditional teict (Weisz: Science 
of Biology , McGraw-Hill Book Company, 2nd ed., 1963) as to their 
effectiveness in independent study. In addition, both independent 
study techniques were compared with th^ traditional lecture methods 
employed in the course. The experimental design was conceived as 
a test of the null hypothesis that "Achievement based on test gain 
scores will show no significant differences 



II 



a . Method 

The material covered by each group was the same: a review 
of mitosis^ a treatment of meiosis, and a survey of fundamental 
Mendelian concepts in genetics. This material is typically covered 
in three 50-mlnute lecture periods and one 3-hour laboratory period; 
the more sophisticated and complex aspects of genetics (physiolo- 
gicai, biochemical, and eugenics) are treated in a additional block 
of three lectures. 
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The treatment administered to each group was a follows: 






# 



Group X “ Independent study using programed text. 

This group did not attend the three initial lectures on 
genetics and was assigned IS lessons in the programed text 
constituting an anticipated 9 hours of sttidy; the rule of 
thumb in this study time is the oft-recited 2 hours study 
for each class hour. 

Group C-1 - Independent study using regular textbook. 

This group did not attend the three initial lectures but 
was referred to the appropriate pages of their textbcfok for 
independent study. 

Group C - Non-independent study. 

This group attended the three initial lectures and used the 
textbook in each student’s customary fashipn. They were 
encouraged not to alter their ordinary learning-study pattei{*n 



The 
Table 1. 



size and composition by ©lass of each group is given in 







Pre test and post test examinations were administered un-’ 
announced just pi^ior to and immediately after the px p er im e n t a 1^^, 
period; the same test was used for bothexamihatidi)^ • The unannounco^ 
testing allowed for an assessment of the announc<i^p examination. 
Performance on fhe genetics portion of the final '^,iiamination was 
tabulated as was information relative to subseque%$ sources of ex- 
perimental contamination (use of program by non-program students ^ 
etc.). Analyses were made only on paired pre test and post test 
scores, thereby eliminating an additional 62 students who had taken 
only one or the other test. 

b. Results 



Table 2 shows the mean (average) pre test and post test scores 
and the mean gain scores computed as the difference between pre 
test and post test scores for each group. (In the table, the sym- 
bol d* = the standard deviation of the mean, a statistical concept, 
describing the spread or dispe;rsion of scores. It may be inter- 
preted as follows: in a normal distribution of scores about 2/3 

of all scores fall within one standard deviation of the mean. For 
example, the pre test score on Group X is 29.6 * 16.7; then one d 
or about 2/3 of all test scores would be expected to fall between 
12.9 and 46.3. The larger*the value of d, the greater the spread 
of scores). 



Inasmuch as the groups were not initially uniform, an analysis 
of covariance was applied to determine the statistical significance 
of differences in the mean gain scores. This analysis was also 
applied within each group to assess differences between freshmen 
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and upperclassmen* The F and P values so derived appear in Table 3 
and show the following resultss 1) Group X is significantly differ-, 
ent from both Group C and C-li 2) Group C and C-1 do not differ 
significantly; 3) there is no significant difference between fresh- 
men and upperclassmen in any group. 



In addition to the data on test scores, the mean number of hours 
of study spent by each group was computed from time records (Table 
2. ) Correlation analysis of study time and* mean gain score showed 
no significant correlation (Table 2) in any group although it 
approaches the 5% level of significance in Group X. 



c . Discussion and Interpretation 



The experimental group using a programed textbook in indepen- 
dent study performed significantly better than either of the control 
groups in learning a given block of material. Although this higher 
test score performance was accomplished by a greater expenditure of 
time, it is to be noted that no correlation was found between study 
time and improvement in any of the three groups. The function of 
study time in learning is intrinsically highly variable at any rate 



It can be argued that had the non-independent study group (C) 
studied as for any exam, its performance would have been better; 
while this is doubtless true, it is also quite obvious that there is 
a marked difference between the active learning required in programe 
learning and the passive learning of a lecture situation. This 
distinction is more striking in comparing the "active” learning of 
the two independent study groups. It is reasonable to conclude from 
the difference in these latter groups that for effective and effi- 
cient independent study, the materials need to be adequately and 
appropriately designed* There is no evidence here to suggest that 
such adequately designed materials need be of the programed or 
"teaching machine" sort. 



The results strongly indicate that this particular pat*t of 
introductory biology can as effectively be taught via independent 
study. The same conclusions were drawn from the earlier Oberlin- 
Earlham study (Kormondy and Van Atta, 1962). There are undoubtedly 
other parts of the subject matter of general biology (and other 
disciplines) which could be similarly treated if suitable materials 
were available; at the moment there are none although several are 
being prepared by competent biologists elsewhere. 



Assuming a somewhat comparable survey of genetics is conducted 
another year, it can be confidently recommended that the material 
on mitosis, meiosis and basic mendelian genetics be self-taught 
through a programed text (or other suitable self-instructional ma- 
terials) and that the three lecture periods heretofore used for this 
material be reassigned* Two lecture periods might be waived in 
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partial eompensatioii for the independent-study time, and the remain- 
ing lecture used to allow more "breathing space" for the more sophis- 
ticated, experimental, and perhaps controverisal areas now squeezed 
into three lectures. Table 1: Class distribution (as percentage of 

total) in experimental and control 
groups. 



\ \ „ „ 


Groupi 1 


N 1 


1 Freshman 


Sophomore I 


1 Junior I 


Senior 


Special 1 

U ^ 


X 


71 


59.2 


28.2 1 


7.0 


4.2 


1.4 ! 


C-1 


86 


75.6 


19.8 


2.3 


2.3 


0.0 

1 * ' 


c 1 

^ U 


116 


32.8 


53.4 


1 8.6 1 


4.3 


1 0.9 ; 



Table 2: Pre test and post test scores, mean 

gain scores, study time and correla- 
tion of study time and improvement! 






Groujl N -I ^Pre Test 



(x +0) 



Post test I Mean jX. xifue 

Gain I 
CT im 




Correlation 
Coefficient of 
Improvement 



X 




::=V 

29.6 


16.7 


70. 5 


20.6 


40.9 


9.4 


3,p 1 


.36 


(NS) 


c -1 


86 


33.7 


17.7 


56.7 


15.6 


23.0 


4.5 


2.4 

t 


-.06 


(NS) 


c ’ 


116 


31.2 


19*2 


59.2 


16.6 


28.0 


6.0 


2.8 


. 08 


(NS) 1 



Table 3i Results of analysis of covariance of 
gain scores (5% re|eotion level) 



iGroup ^ I 


F value | 


Significance 


X vs C-1 


30.26 ° 


P .0001 


1 

iX vs C 


18.80 


P .0901 






*** 1 


X vs C-1 

1 1 


2.74 


■ 1 


i 

X: freshmen vs 


1 * » 




upperclassmen 

1 ' 


2.89 


NS 


C-1: Freshman vs 




• 


Upperclassmen 


0.04 


NS 


|c: freshmen vs 






1 Upperclassmen 


1 0.88 


1 NS 
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d. Student Evaluation 



r?® the flrat experienue with programed instruction for two 

aSiv^L experimental group. They reacted favoj? 

ably to this method of instruction. Forty-four thought it was a ^ 

“ positive than nega- 

tlve th an negatIvg )reactioni while eighteen had a negative reaction. 

The students were evenly divided on whether they preferred 

°I! * PJ^ogram for presentation of material. However, with- 

preferred programed materials to 

the use of a regular textbook. ‘ 

A majority considered that what they learned from the program 

spout on it. They felt the program gave them ’.a 
It ^ direction, the pace of the program was about right most of 

the time and the step size^ of the program was about right most of 
|tne time. 

Thirty-four of the students indicated they could easily review 
the programed materials while ten said they couldn't and 17 were un- 

CSX^TAXXl e 

characterize programed materials as they exper- 
ienced It, the following responses were checked ifjost often. 

No . Responses 



36 

35 

29 

23 



It was mechanical 

In sj^lte of its disadvantages, I leappod much 
It gave me a feeling of making programs 
It made me think ’ 



In the group using the textbook, the most frequent responses to 
me think°-^37 follows: mechanical - 29, iitppersonal - 27, madj 

We see from this comparison of methods of instruction that the 
personal touch of a good instructor is still appreciated. by students 
(nore than a program or text . 

1.1 asked how programed instruction should be used in college. 

Iti a supplement to textbooks and lectures”, 35 responded 

to bring students up to the starting level in the course”. Only 6 

thought programed instruction should be used as a primary teaching 
ievlca In place of lectures-. -- i^eacnxng 

In summary, about two-thirds of the programed text group (Group 
favorably to that approach although they were equally 
livided as to their preference of lecture vs. programed material. 
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and impersonal aspects of the technique, 
hich many felt to be the case, most fouvid that it gave a feeling of 
progress and of accomplishment. » » « reexxng or 

textbook group (Group C- 1 ) was less 
A 4 .^ inclined toward that approach (somewhat over half) and 
? i*®® positive of its effectiveness. The reacions of 
both subdivisions of the traditional group (G) are* in general 

their^oLtlcul!®* are mor^- favorable toward 

tneir particular method of presentation. 

«• Instructor Evaluation 

indicated tha j: he had _used programed material 
aIa aI tlme.^ Heparticipptfed'iT^Swi--^ the o'iigTn^aX'l^lQeafr^ 

1960 ^ programed material in general biology in 1959 and 

4 

5° evaluating these materials, for GLCA, he considered hin 

*°"®*’‘* programing. He said, "Hot in general, 
ut xinder specific instances, not as a substitute for teaching;" ’ 

of *'** the content justified the use 

?, T° program does save instructor's time and its use 

justified in terms of its cost, at least by curppnt book prices. 

♦t. 1 . I'i his overall reaction to this programed material at 

?"? positive nature. He would like to use 

similar material again. K 

It .» London, Department of Psychology* Denison 

a research project using the '^commercial pro- 
^ ALl§Ay®\g.-Of Behavior Parts I and II, by Holland and Skinner. 

a. Method 

1 -h. designed to test differences in achievement using 

the adaptation of the text in 

to fiir Th^ P*'®®® ®*y^® hlanks 

programs were the same. The pro- 
grams were the primary method of instruction during the/"studv. 

Approximately one hundred students used the published text while the 
same number used the adapted text in a unit which is part of the 
freshmen psychology course. 

In tha fall of 1964, a partial field study of the coramerclallv 

Sklni« ?-h; h^slvsis of Behavior (Holland and 

Skinner, B.F., 1961; was conducted at Denison Oniversity. This text 
18 a linear program, wherein the student is presented with a fixed 
sequence of stimuli, to which he ttespondw with a constructlon-type 
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(f lll-ln-the-blank) answer. The format permits the student to com- 
pare his response with the correct answer which appears on the next 
page and precedes the next stimulus panel in the sequence. 

The stimulus material employed in this study was part of the 
assigned reading in the Introductory Psychology course. Students in 
six sections of this course served as subjects. Half the randomly 
selected students in each section were instructed to study Parts I 
and II of the programed text» pages 1 through 71, The remaining 
students were given an adaptation of this material which presented 
all the concepts included in the program in a conventional t.ext- 
bo'ok format. 

! 

The adaptation was constructed by essentially eliminating much I 
of the redundancy present in the programed text. The blaiik spaces 
in each item were replaced with the correct words or phrases present- 
ed in the program. These were underlined. Thus, this format did 
not elicit any answer-type responses from the student. The number 
of items evoking the same response and the number of examples illus-j 
trating a concept rcere reduced. Information from exhibits preceding 
the sections of the program was interspersed at appropriate places 
in the text of the adaptation. When necessary, the order of items 
the program was changed to permit greater continuity. The eleven 
sections thus constructed v?ere reproduced on fifteen single-spaced 
mimeographed pages. 






All materials and instructions were distributed during the 
first class meeting of the semester. ^*s assigned to the piogram 
group (Group P) received the following""instructions : 

This semester the Psychology Department is Conducting a partial 
field study of the programed text, The Analysis of Behavior 
which is part of the assigned reading in Psychology 101 . Thi§ 
study is sponsored by the Great Lakes College Association Pro-- 
gramed Instruction Project. 

An adaptation of Parts I and II of the programed text will be 
presented to about half of the students enrolled in this coursel. 
This adaptation presents all the concepts included in the pro- 
gramed version in a conventional text book forij^at. 

Participation in this study will require very little special 
effort on your part. All students may purchase the text, and 
all students are responsible for the assigned material. How- 
ever, those students receiving the adaptation will be instructs i 
not to study or look at Parts I and II of the programed text. 
Those students who do not receive the adaptation are instructed 
not to study or even look at this adaptation . Remember: study 
Parts I and IX of the'programed text exclusively. Do not study 
the adaptation. Once again, rest assured that the programed 
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toxt and its adaptation are merely two methods of presenting 
exactly the same material* 

You will be tested on all this material as part of the regular 
testing sequence in this course. A half-hour multiple choice 
quiz, cbverlng the material in this adaptation, will be given 
eight days hence. 

In addition to the mastery test scores, submitte^^nfpjpjinaj* 

tion on time spent studying the material. These data w^e gathereT 
following the test administration. General instructions were as 
follows: • 

m 

Your participation in this study imposes no restrictions or 
limitations on your amount, method, and pattern of study. How- 
ever, we wouM like you to keep track of the amount of time you 
spend on this material, according to the instructions on the 
following page. Please be honest and accurate in recording * thife 
information. Whether you study a little or a lot will have no 
direct bearing oh our evaluation of you as a si^udent. Your 
final grade will reflect your performance in labs and on exams 
exclusively. 

The ins trucl^ions for time keeping provided for the recording of 
ten separate study sessions of each of the eleven qedtions. S's wer< 
informed that they might study each section as freqq^ntly as ^hey 
wished and that the sections need not be studied ah;';|^qual number of 
times. Conventions were established for timing part^pl readings and 
reviews. When, for any reason study times were estimates, ifLs were 
instructed to record such estimates according to a scheduFe repro- 
duced on the instruction sheet. 

All instructions for ^'s in the adaptation group (Group A) 
appeared at the beginning of the adaptation. These were identical 
to those given to Group P, with one exception; s were infov^acid that 
they could purchase the programed text, but werK caht ioned to avoid 
reading sections 1 through 11. All materials were collected follow- 
ing administration of the mastery test. Approximately 90% of the 
students returned the adaptation and/or time keeping sheet. 

The mastery test consisted of '33 multiple-choice questions, with 
sampling from all of the 11 sets or sections included in the assign- 
meht .The test was during regularly scheduled class periods 

s were allowed the entire 50 iininute'* period in which to complete 
the test. ^*8 were instructe-d to answer every question* T-he test 
score was the total number of correct, responses* 

b. Results 

Two measures were derived from the present study: (1) mastery 
test sdore; and (2,) total study time in minutes. The mean difference 
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in test pepforinance between groups P and A was evaluated by use of 
the t-distributlon. Differences between treatment groups with in- 
structor held constant were similarly evaluated. Means and standard 
deviations and t-tests, are presented in Table 4 below. As indicated 
in the table • none of the differences in test performance were sig- 
nificant. 

Table 4 s 

Means • S.D.*s and t-tests for Mastery Test Performance 
Groups P and k 


















Instructor^SKoup P 



Group A 



t Tests 









1 S « D 


” 1 


r~x 1 


1 S.D. 


t 1 


1 


r~ p 


1 1 


31 


25.5 


1 


38 


26.4 


r 4.02 


.95 


67 


II . s . * 


2 


52 


24.9 


3.68 


* 50 


25.6 


4.22 


.90 


100 


M.S. 




“'1 


i 


4.55 


20 


24.8 


3.55 


.95 


39 


M.S. 


IVll Sections 


















104 

i ■ 1 » ^ 


25.3 1 


3.91] 


108 1 


1 25.7 


4.00 1 


1 .74 1 


1 210 1 


N.S. 



Analyses Identical to those described above ifere applied to th 
tudy-time data«- Table 5 summarizes these respltg'i: Despite consid-? 

rable skewness in individual distributions and apparent heter 
f variance between groups, no transformations weip applied to 
iata* As indicated In the table, however, all of the mean dif: 
snces were statistically significant. In each case, p*s i 
studied significantly less, on the average, than did Group 



these 
of the mean differ- 
case, P*s in Group A 
P S»8. 



Table . 5 : 



Means, S.D.'s and t-tests for Study Time Data 
Groups P and A 



[nstructo r Group P 



n 


[ Meani S.D. I 


1 n 1 Mean 1 


PS.D. 


t 


fd*f .1 


— — 


1 1 


24 


224.4 1 106.4 


37 1 141.5 


[,•81.6 


' 3.25 


\ 59 


.01 


2 


30 


201.1 1 68.1 


47 I- 127.3 


'65.1 


5.47 


95 


. 01 


• 3 ' 


20 


259.8 156.8 


’ 20 142.6 


52.'-'4 


3.17 


-38 


• 01 


'^otal 1 


1 94‘-| 


219.6 1 104.0 


pT04 1 135.3 1 


Heo.o 1 


1 d.64| 


r 1 .01 



- c* Discuss ion 

One major result of this study iS| that no difference in mas- 
ery was found bstween the group (P) instructed to study a specified 
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portion of the programed text and the group CA) presented with identl4 
cal material in a standard textbook format. This result lends support 
to the ^thesis that elicitation of an overt or covert response during 
study 18 not a necessary condition for subsequent demonstration of 
mastery, pis conclusion of course, is based on a comparison of grouj 
means. TMs independent-groups design does not permit analysis of thJ 
pdividual cap. pat is, we have no opportunity to compare the test 
prppance of a given subject on different material presented with 
bop formats, op which elicits specific responses (program) and one 
pich does not (adaptation). Analysis of this question requires a 
sub^eps-as-own-controls design with independent procedures for 
pasting difficulty level of both stimulus materials and mastery test. 



. The second h^oteworthy result was that significantly less study 

time, on the average, was spent by S' s' in Group A than Group P. With 
pudy time as a criterion, it seems reasonable to conclude that the 
pespt adaptation was a more efficient method to achieve a given" 
level of mapery. However, generalizations beyond the specific para- 
aigm and stimulus materials employed in the present study seem un- 
Wpranpd. Spection of different stimulus materials might result in 
ppifpant differences in test performance (in either direction) and 
pduction or reversal of the study time effect. 



This study time differential seems attrlbutab|,f^ to a number of 
ractors , working singly or in combination. The present research was 
pot supiciently detailed to determine which of thea<| factors was 
perative. The most obvious interpretation is that f he time differ- 
ential reflects the fact that S's in Group A had approximately half 
pe pount of reading required for Group P S's. Thal^is, the result 
nay bs a dlreclr rssullr of the rsducsd :?©dundancye 



■ An alterative explanation suggests that a considerable portion 
df the time difference may be attributable to the greater compactness 
6f the adaptation (15 pages versus 71 for the program). Use of the 
prmer would involve less sheer page-turning time and fewer anticipa- 
tory attending responses. That Is, the difference may be due to a 
rss-tiian-optimal program format. Related to this alternative is the 
pcsr^ility that the difference reflects the relatively greater 
Efficiency with which sections could be reviewed by S's in Group A. 

Ihe result would be attributable to differences in review time, rather 
than in initial reading or study time. Finally, the observed i 

difference may be due to the fact that S's in Group P were required to 
select a response, construct that response and compare it with the 
correct answer. Mo such operations were performed by Group A. The 
x*esult may therefore be a function of differential response time* 



These results suggest, that further research in which the entire 
programed text. The Analysi s of Behavior, is adapted as above may 
serve to isolate the variabie(s) responsible for the reported time 
difference* Indeed, the conversion of linear programs into more 
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conventional formats and subsequent field tests thereof may lead to 
the isolation of critical variables in the writing of maximally 
efficient programs and/or conventional text books* 

6. Professor John Reinheimer, Chemistry Department, College 
of Wooster directed a research project using the commercial programl 
Pgogyawed Sup p le ments for General Chemistry Volume I and II* bv 1 
Barrow. j 

a. Method 

The study was designed to compare achievement on a unit exam- 
ination; one group used the programed materials while a second groui 
received individual help by the instructor. The instruction for 
both groups was supplemented byL_reg^ular classroom instruction. 
Approximately 15 subjects were in each of the two groups « 

At the end of the first unit of work in the Freshman Chemistrj 
course, the students with the lowest thirty scores on the unit t^*t 
were selected for the study. Approximately five sections of the 
course were involved so students were matched aco|drding to the 
scores on the first unit test and by sections* Approximately two I 
weeks before the second unit test, students in 6|*oup A were given 
the opportunity to use the programed materials covering the content 
of the second u^it for review purposes. Those in "|roup B were givei 
the opportunity' and encouraged to come in to get^ 4 i|Vtra help from th< 
instructor and to prepare for the second unit tes^* 

Two weeks before the unit test number threi|| the treatment wac 
reversed* Group A students were encouraged to g^.lj extra help from 
the instructor and students in Group B were given the programed .I 
materials to study in preparation' for the exams*' 

Test scores on test 2 and test 3 were compared for each match- 
ed pair to see which treatment resulted in greater achievement* 

The ^ time spent on the programed materials and time spent receiving 
additional help by the instructor were recorded for each student in 
the study. 



Student Evaluation 



Eighteen out of twenty-two students indicated that they had 
never used programed materials before* The overall reaction to us- 
ing ^programed material in this course was favorable for 16 students, 
indifferent for 4 and unfavorable for only two. 



The group divided evenly on preference for programed materials 
or regular textbook. The preference for lectures rather than pro- 
gramed materiaie wee an overwhelming 13 to 1. Five were indifferent^ 
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Most of tho students indicated that what they learned from the 
program was worth the time spent on it. They felt it gave them a ! 
good sense of direction and the pace of the program was about right 

most of the time. The majority also felt that material could easily 
be reviewed. ^ 



When asked to characterize programed material as they exper*^ 
ienced it, the two responses checked most frequently were as follows: 



No . 

1-0 

9 



Response 

It gave me the feeling of making progress ' 
It speeded up my learning 



^ The students felt that the best way to use programed instruc- 
tion in college was as a supplement to textbooks and lectures. The 
second choice was for remedial work when the student falls behind. 

Seven students indicated the use of programed materials favor- 
ably affected their attitude toward the subject matter of the course. 
Thirteen said it had no appreciable effect and only.^one said it had 
an unfavorable effect. 



c. Instructor Evaluation 

/ I 

The instructor had never used programed materials before. The 
program was used for remedial work when the student? felrlr^^l-nd . It 
was also used for supplemental drill. 

The instructor *s Initial attitude toward programed Instruction 
was favorable. His comment on Barrow was the following, "Barrow is 
very good.” He found no errors in the content and no technical 
errors in the program. The instructor felt that the Importance of 
the content covered by the program justified .the amount of time re- 
quired by the student. He indicated that the program saved the in- 
structor time and the program was justified in terms of its cost, as 
well. He would like to use similar’ material again. 

7. Professor Fernando Reodriquez, Department of Romance 
Languages, Ohio Wesleyan University conducted a research study in his 
first year French course using thQ commercial program, TEMAC French 
Phonetics , by Burroughs. . — ' * 

a. Method 

The study involved supplementary use of the material including 
tapes versus the use of a basic text In French which includes tapes 
for use in the language laboratory. 

.. The experimental group was a class of seven students wiio took 
the course In the and used a text which did not have 
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accompanying tapes* The TEMAC programed text and tapes provided the 
supplementary phonetics study which the students needed. A compar- 
ably sised control group was randomly chosen from those students tak 
ing the course the fall semester of 1964. This control used a basic 
text which included the tapes for phonetic drill* 



A tape of conversational French which students were asked tc 
comprehend served as the pre and pos'l' test for both groups. The ex- 
periment for each group' lasted six weeks* 

i 

b. Results 

The comprehension of the tape of conversational French which 
served as the pre and post test was evaluated subjectively by the 
instructor, using a 10-point scale. 

E Pre test Mean Post test Mean Gain Scort 

Mean ' 

7 3.6 6.3 2.7 

7 3,6 6.9 3 .’3 

Both groups showed improvement over the six weeks period from 
using individual instruction through programed tspes or regular 
language lab* However, there is no appreciable difference between 
the two groups ijii the amount of improvement as meipured by mean gain 
scores. ' 

■i 

c. Student Evaluation 

In the experimental group this was the firs| experience with I 
programed material for six of the seven students*.' Four expressed ah 
unfavorable overall reaction to the programed materials used in the I 
course, one reacted ‘favorably and two were indifferent. I 

The students believed that the place for programed material in 
college was for remedial work when the students fell behind. Five 
of seven checked this response* 




Group 

Program 

Control 



As a way of learning, five of the group felt that programed 
;riaterial was less effective than a textbook or other materl,als they 
liad used. The students worked with the programed materials an aver- 
age of 13.5 hours. 

_ ’’Compared with what you learned; from the program, was. it worth 
, srour time?”- Three , answered no, three were, uncertain and only one 
: said yes .... 

-comiiinients students .made , on the back .of their ques'*'* 
^ionnaires indicated that the programed materials were not 
Rhe beginning French student. 



( 
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InstPuictor Evaluation 



Rodpiquez had not used ppogramed matspial befope* He indicate<i 
that his initial attitude towapd ppogpamed instpuction priop to test-^ ‘ 
ing the matepial was favopable. He went on to evaluate in his own 
wopds thus, ”I expepimented with the TSMAC ppogpain in Fpeneh 
.Phonetics this suttiwep because the textbook used did not provide tapes 
for language lab use. l thought that this programed matarial would 
be of some help to the students since the introductory booklet claiiii** 
ad that the program was designed for the beginning student as well as 
the advanced student. The results of the experiment were disappoint- 
ing for the following reasons: 

(1) The program content deals with phonetics whereas the ideal 
program for the elementary class should be on pronunciation. 

(2) The student was required, to learn the phonetic symbols 
which added to the already heavy load of new material to be le-'*r,ied.. 

(3) The time allowed on the tape for repetition and responses 
remained the same even though the material increased in difficulty. 

(4) The students expressed difficulty in making an association 
between the programed work and the work in the clasqroom. 

(5) The arrangement of the programed material itself is faulty 
in places and especially confusing at the beginning’, 

In general, the results of the TEMAC material with an elemen- 
tary class were unsatisfactory; however, I feel the -program would be 
more successful when used by an advanced student working independent- 
ly or in conjunction with an advanced class in cohversatioh or 
phonetics . ” 

i 

8. Professor Ruth Smyth of the Mathematics Department, College 
of Wooster, directed a research project using the commercial program, 
TEMAC Analytic Trigonometry . 

a. Method 

The study was designed to test if there was any difference in 
achievement when the primary means of instruction was by programed 
I materials or by conventional lecture and textbook. The period of the 
I study was the entire first semester and involved two groups of '25 
I students each using programed materials exclusively , and two groups 
j of 25- students each using instruction by large class lecture and 
I small discussion sections. The students were chosen fot the study 
!on the basis of the mathematics placement test administered at the 
beginning of the academic year* Students scoring below a given cut- 
off point were recommended for the course in Analytic Trigonometry* 
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The students in the four groups were matched as closely as possible 
on the basis of placement test scores^ sex and year in college* 



The two experimental groups were given the programed text and 
told to work through the program to a specific page each week* Thin 
work was to be done outside of class with no help by the instructor • 
Bach week a quiz was given on the material and the program assigned 
for that week* The two control groups met together for mass lec- 
tures two periods a week* The third period the two groups. met 
separately for discussion with the instructor and then a quiz was 
given* Achievement was measured by weekly quizzes • hourly exams • 
and final exams* The same quizzes • hour tests and final examina- 
tions were given all four groups* 

b* Results 




Group 

Program 

Lecture 



M 

48 

47 



Mean Gain Scores 

17*16 

8*96 



Standard error of Difference 
t-raflo based on Independent Samples 



7.67 

1,17 N*S* 






*ne cata xnaxcare max tnere was considerable difference in the 
mean gain scores of the two groups favoring the 'group which used th 
program* However^ there was so much variability! in scores within 
the groups that the difference in mean gain scorgip between the con«r 
trol and experimental groups was not statistically significant at 
the *05 level* 

c * Student Evaluation 

This was the first experience with programed materials for 41 
out of 48 students in the experimental group* When asked what theli 
reaction was to the use of programed material in this course, 16 
gave favorable responses while 28 expressed a negative reaction* 

There was a 2 to 1 ratio favoring the. use of a textbook rather than 
programed materials* The ratio was 3 to 1 favoring the presentation 
of material by lecture rather than programed instruction* 

When asked if they considered what they learned from the pro- 
gram worth the time they spent on it, 17 said yes, 19 said no* One 
of the major complaints was the difficulty of reviewing the materia] • 
Only 12 thought it was easy to review while 31 said it was not. 

The students were asked to characterize the programed materia], 
ae they experienced It. Three responses were checked more than 
twice as often as the others* These were as follows: 
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No. 

40 

32 

31 






Response 

It was mechanical 

It did not hold my attention 

It ‘was boring 



The responses oheoked most often to the question, "As you see it now 
how should one use programed instru(^tion in college? were as foll- 



ows: 



No, 

32 

28 

22 



Response 

As a supplement to textbooks and lectures 
For remedial work when tne student falls 
behind 

To bring students up to the starting leve, 
in the course 



Half of the students responding indicated that the use of pro- 
pamed materials unfavorably affected their attitude toward the s^b- 
ject matter while only 5 stated it had a favorable effect. Eichteen 
said it had no appreciable effect. 

In contrast, the group receiving the lectures were 50-50 in 
their reaction (favorable versus unfavorable) to this method of in- 
struction. A large majority felt the method justified the time 
spent on it and the material could be easily review!^. 

However, when asked to characterize the methods used in the 
course, two of the same three responses led the li||. 



No. 

40 

31 

20 

20 



Response 

It was mechanical 

It was boring 

Made me think 

Did not hold my attention 



Seven students said the lectures had a favorable effect on thei 
attitude toward the subject, 15 said they had an unfavorable effect 
and 21 said they had no appreciable effect. 

d. Instructor Evaluation 

^Professor Smyth indicated that she had^ever used programed 
material before. In her own words her reactn^ to the evaluation of 
the programed material was as follows: ”The Material was poorly 

edited. There were pany errors in the answers. There was no way for 
students to know what was important. No index was provided and too 
much 'spoon feeding*^" 
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She went on to say that the importance of the materi al AiA 
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justify the student s spending the amount of tileejoiiitrequired of the 
program. The program would save the instructor time if he were not 
teach as well as test th-^ material. But she thought the 
students learned only when there was some teaching done in addition 
to using the program. 

Liere, with a senior psychology major 
student by the name of Miss Barbara Arnold, evaluated the effective- 
J?!? programed instruction using the Holland-Skinner program en- 
A.Paly®is of Behavior. The program was tested as a primary 
teaching device and as a supplementary teaching device. 

a. Method 

Two experimental groups, each having between 30 and 35 subiectc 

selected. These groups were select- 
ed by ordinary college sectioning procedures. They were controlled, 
however, on the basis of aptitude test scores. Most of the students 
were freshmen or sophomores. 

experimental group used the program only. The secoa4 
experimental group heard lectures on the material and used the pro- 
gram as well. The control group heard the lectur^p only. ' 



wu PT^ogrma were restricted at the Departmtnt of Psychology 

students who used the books. The students were required to use it or 
certain days at certain times. vT ® ^ 

0 

tests on the content of the material were used. 
Miss Arnold kept a record of the use of the programs. 



Results 



Group 



N 



# 

A’ 



1 Lectures only 34 

2 Program only 30 

3 Program and 

Lectures 24 



Mean Gain Scores 

21.52 

30.71 



30.10 



Difference in Means: Groups §2 and #3 

Standard error of difference 
t-ratio based on independent samples 



.62 

1.54 

.40 



NS 



Difference in Means: Groups #2 and §1 -9.20 

Standard error of difference • i.04 

t-ratlo based on independent samples -8.65 p<<.01 



Difference in Means: Groups #1 and #3 

Standard error of differences 
t-ratio based on independent samples 



“8.58 

1.48 

-5.80 p^.Ol 




The data indicate tha«t while students in all groups learned 
during the study • the students in Groups #2 and ^3 using the program] 
learned significantly more than the group receiving lectures only. 



Student Evaluation 



For three-fourths of the students in the two groups using the 
programed materials* this was their first experience with prog*ramed 
instr’uction. Their reaction was quite favorable. Sixteen Indicated 
that the use of programed materials was a very positive learning ex- 
perience* and they would like to use them again. Twenty-six ex- 
pressed more positive than negative reaction to it. Only three ex- 
pressed an unfavorable attitude. 



The students were asked* **Without an instructor* would you pro* 
fer a program or a t^axtbook?” Thirty-one preferred programed mater- 
ials j only 10 preferred a textbook. 

The students who used the programed materials were asked to 
compare this method of presentation of material with the lecture 
method. Twenty-three preferred the programed materials while 13 
preferred lectures. 



"Considering what you learned, was the program worth the time 
you spent?" Thirty- three said yes* only 5 said not The students be- 
lieved that the program gave them a sense of direclion most of the 
time and the pace was about right moot of the time, 



The question* 'Could you easily review the programed material?’! 
did not bring favorable responses. “Nine said "Yes^** 16 said "Mo", 
while 23 were "Uncertain." 



With the following two questions students were asked to 
characterize programed material as they experienced it. They could 
check more than one response. The three positive and three negative 
responses which were checked most often are presented below. 

Positive responses 

In spite of its disadvantages* I learned much 
It gave me a feeling of making progress 
It made me think 



Wo. 

39 

23 

17 





negative responses 


28 


It was 


mechanical 


25 


It was 


boriag 


16 


It was 


impersonal 









The students were asked how they thought programed instruction 
should be used in college. Both groups which used programed mater- 
ials (one exclusivelyt one in conjunction with lectures) felt the 
best way was as a supplement to textbooks and lectures. Thirty-one 
checked this response. The next highest total, 26, was "to bring 
students up to starting level in the course." Only 7 thought it 
should be used as a primary teaching device in place of lectures. 

d. Instructor Evalivition 

Professor Van Liere indicated that he had never used programed 
material before. Prior to this evaluation, however, he expressed an 
unfavorable attitude toward programed instruction. 

His comments following the evaluation were, "I have a negative 
reaction to programed material, especially untested material, for 
college students. There are unwarranted assumptions as a basis for 
programed material: 1) That reinforcement techniques developed 

with infra humans apply directly to humans, 2) Tl^at the 'reinfor- 
cers' integrated in programed instruction are adequate for all 
college students." 

He found no errors in the content of the program nor in the 
mechanics of presenting the material. He did not feel that the im- 
portance of the aontent justified the amount of ti^e required of the 
student, and the program did not^ jiave the instruc||r time. Also, 
the program did not justify its cost. 

Barbara Arnold participated with Professor Van Liere in the 
evaluation of the program and filled out an instructor questionnaire. 
She had not used programed materials before this time and her 
attitude toward programed instruction prior to the evaluation for 
3LCA was also unfavorable. 



j Her responses to the questions paralleled those of Professor 

Van Liere and she made these specific comments regarding the program , 
r'This type of programed material which requires nothing more than 
token responses is very boring. In general, our students needed to 
pe prodded a bit for the last week, or whole last half, of the pro- 
gram. Besides not retaining significantly more of this material than 
|the lecture group, the book groups spent three or four times as loug 
jin acquisition, which hardly seems justified. The largest single 
problem involved is, I feel., maintaining students' interest and 
notivation." 



In summary, the reaction of both evaluators to the use of pro- 
gramed materials in this course was more negative than positive. 
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10. Professor Robert 0. Weiss, Department of_^eech at DePauw 
University, evaluated two programs, Lehman ! s j*‘aTr^i^me^ Procedure “ 

(Tutor text) and Gray and Rea* a^ Parliamentary Proce dure ( Scott Pores* 
man). — — . — — « 





Method 



He used approximately 10 students in each group, 
were matched according to their sex, their verbal score 
and by grade point average, 
seniors in college. 



The groups 
on the SAT 



Almost all cf them were juniors or 



Both groups were treated alike. In administering the program 
no control was kept over the amount each program was used. They 
were motivated by fairly frequent mention of the program in class. 
The students were informed that the test was constructed without 
specific reference to cither program. The pressure of other worTc 
was likely to preclude the use of both programs by one student. The 
unit took four and one half weeks. All students were expected to be 
familiar with Robert's Rules of Order . 

Instruments used were a pre test and a post test constructed 
by Weiss, the Student Evaluation Questionnaire and the Instructor 
Program Evaluation Questionnaire. 

b. Results 



The pre test and post test used in the evalu#tion each had 30 
true-false questions and 15 multiple-choice covering parliamentary 
procedures. The true-false questions counted one point and the 
multiple choice questions, 2 points. The tests we^# scored on a 
right-minus-wrong basis. The students were told nol to guess. The 
test results for the two groups are given below. 
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Group 


Pre test 




Post Test 


Gain Score 


A- Using Gray and 


Rea Mean 


-1.1 


24.2 


25.3 


Program 


N 


16 




B- Using Lehman 


Mean 


r6.8 


29.2 


36.0 


Program 


N 


18 








No further statistical analysis was done on the data* but the 
mean scores for the two groups indicate that the students using the 
Lehman program learned more of the material that was included in the 
tests. No information is availaiile on the amount of time students 
spent working on the programs. 



c. Student Evaluation 



Half of the students indicated this was their first experience 









with using programed materials* The overall response to the. program 
was favorable since 70% of the students indicated it was a very posi 
tive or more positive than negative learning experience* Aleot a 
large majority, 62% of those responding, answered yes to the questio 
"Considering xhat you learned, was the program worth the time you 
spent?" 

In response to the question, "As you see it now how should one 
use programed instruction in college?", the following interesting 
contrasts were observed* The students were allowed more than one 
response to this question* 

Hfo* Response 

24 As a supplement to textbooks and lectures 

To teach material that the teacher does not want to 



The results Indicate students see value in programed materials 
for supplementary purposes but not as a primary teaching device* 

Twenty students indicated that the use of programed materials 
favorably affected their attitude toward the subject matter of the 
course* Only two responded unfavorably* 

d* Instructor Evaluation 

The instructor indicated he had used programed material before 
in the course named Public Discussion* He was using the present pro** 
grams to teach material that he did not want to cover in class* He 
described it as follows, "It gives a more digestible approach to 
material contained in regular reading*" His initial attitude toward 
programed material was favorable* 

His specific reaction to it was as follows: The programs 

essentially covered the same material he wanted* He found minor 
errors in the content of the program and some technical errors in thu 
program* The instructor was not certain that the importance of the 
content covered by the program Justified the amount of time required 
by the student* He did not believe that the program oaved the in- 
structor time, but that it was justified in terms of its cost* 

* 

11* Profes.sor Joseph Vetmore, Department of Education, Ohio 
Wesleyan University, conducted a study comparing two commercial prow 
igraned texts in statistics* The first was Statistical Measures bv 
fOorcw and the second book was Basic Statistical Cooceots Tv Bradlev 
!and McClellands — 



8 

3 



cover 

As a primary' teaching device in place of textbooks 
Afj a primary teaching device in place of lectures 
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£i» Method 



The etudly Involved two classes of 40 students each. All the 
work on the progran was done outside of class with no help frow the 
instructor* The study lasted approximately one week* The same 
statistics achievement test was used for the pre and the post test* 
Gain scores were used to compare the achievement on the two pro- 
grams* The amount of time the students spent working on the pro- 
gram was recorded* 
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b. Results 
Group 



Program Gp |1 - Bradley 
I and McClelland 

Program Gp 
Goro* 



N Pre 


test Means 


Post test Means 


• 

• 

Gain Scori 


34 


11.3 


24*1 


Means 

12.8 


26 


11.1 


27.3 


16.1 



These preliminary results indicate that the students learned 
a considerable amount of material from both programs* The statist! 

t eal analysis t (60) » 2*33 indicated the group us^us The Gorow text 
ad a significantly higher mean gain scores (P = 1*05)* ..The possiblje 
core on both pre and post tests was 40 points* 



c * Instructor k ' .valuatioa 

j Prefessor Hetmore had used programed texts hfi'bre* He had 

Used the text by Bradley and McClelland in the sa«e course* He in- 
dicated that his attitude toward the programed in|ibruction material 
fras enthusiastic* ^ 

4 I 

. In reference to the book by Gorowy Wetmore said the following » 

*At the first glance I was not Impressed with this book, it seemed 
rather skimpy, but it does provide good material. I like the fact 
that it is a branch rather than a linear text. The biggest problem 
is that students read this as a text— page by page «- even when i 
instructed not to* They do not let the book teach them, but attempt 
to memorise the material on each page* I feel that I probably would 
use this book next year!”. As far as he could tell there were no 
errors in the book either of the technical or content kind. He felt 
that the importance of the content justified the amount of time re- 
quired of the etudent* Also the program saved instructor time and 
It was justified in terms of cost* in summary, Professor Wetmore 
taid that his overall reaction to programed material at this time 
was very positive and that he would like to use it again. 

Referring now to the text by Bradley and McClelland, Hetmore 
md thie to say, have used this satie programed text before and 
tad tn iuppleiient it with lectures » I would still hare to dlear up 
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understandings since students do not know how to use this material 
even after being told.'^ He found there were no errors either in 
content or technical errors of programing* Even with this program 
he felt that he preferred the programed material to the teietbook an|l 
to the lecture but that he would have to supplement it* 

d. Student Evaluation 



This was the first experience with using programed material 
for 30 out of the 34 students using the Bradley and McClelland text* 
The overall reaction to this text was favorable for three-fourths o? 
the students* The large majority of the students indicated* that 
without an instructor they world, prefer to use the program rather 
than a typical text, but regular lectures were preferred to the 
program* 



The time spent on the program was justified in terms of what | 
was learned in the opinion of two- thirds of the group* Students | 
claimed it speeded up their learning and made them think* A large ' 
majority indicated the best way to use programed instruction was as! 
a supplement to textbooks and lectures* Two- thirds of the group I 
felt the use of programed materials favorably affected their atti- 
tude toward the subject matter while the rest of the group claimed. I 
it had no appreciable effect* 

This is also the first experience with programed instruction 
for 24 of the 28 students using the Gorow text* Eighty- four percenl; 
of the group had a favorable reaction to its use* Without an in- f 
structor they preferred the program to a regular text* The group I 
was evenly divided over whether they preferred l<)!ctures or the pro- 1 
gramed material* 

Sixty-four percent of the students indicated the amount of I 
learning justified the time spent on the program* In characterizing 
the program the group indicated it gave them the feeling of making T 
progress* When asked how programed materials should be used on the! 
college level, the leading responses were, as a supplement to text- I 
books and lectures, and for remedial work when a student falls be- 
hind* Sixty-percent of the students indicated the use of programed 
material had no appreciable effect on their attitude toward the 
subject matter* 



12* Professor Dorothy Whitted, English Department of Ohio 
Wesleyan University, conducted a research project using the 
commercial program. Effective Writing by Smith and Stapleford* 

a* Method 

The study was designed to test the effectiveness of the pro- 
gramed materials versus tutorial help by the instructor in helping 
students bveroome deficiencies in English composition* The student 

— ' — 4 ^^ 




included in the study were those placed in the £.iglish proficiency 
program for remedial work on tlwr -recommendation cf the English pro- 
fessor* The student remaxned xn the program until a proficiency 

test was passed at which time che student returned to the regular 
program* 

Pairs of students were matched for this evaluation study on 
the basis of their errors in composition on an essay they were re- 
quired to write when they first entered the proficiency program. 
Approximately ten students were given the program materials and 
about ten were given tutorial help* When each student felt *he had 
overcome his weaknesses « he was given another essay to write which 
was scored for errors on the same basis as the first essay* 
Achievement for the two treatments was based on gains in .scoires on 
the two assigned essays* VJL. 

Professor Whitted used the same commercial program in -mothei 
study involving two classes ^ each of twenty students, in a regular 
freshman English course* During this two week experiment one class 
used the programed materials exclusively while the control group 
class used the regular classroom instruction. A pre test and post 
test consisting of a required written essay was used to compare i 

achievement in English composition under the two methods of instruc 
tion* ^ 



b* Results 

1 * The score on the pre test and post test essays 
was a function of the number of sentences in th^ljissay and the num-i 
ber of different kinds of errors that were made'|ri the composition*! 
The actual score was the ratio of number of sent|nces written per 
kind of error* For example, a score of 4*5 indicates that the stp^ 
dent made an average of one kind of grammatical error in every 4*^ 
sentences that he wrote* The total number of errors in any essay ' t 
was not computed since the main concern was the types of errors I 
that were made* There nan an average of two months between the* prel 
and post test essays* 



Group 


' 




Pre Test 


Post test 


Gain 


Program 


Mean 
Mo * in 


Gp 


6*6 

7 


4*9 


-1*7 


Tutorial 


Mean 
No* in 


6p 


• 

9*0 

7 


4*6 


-4*4 



I Each group wrote an average of approximately 20 sentences on 

both the pre test and the post test* Both groups did poorer on the 
post test essay in regard to. the number of different types of errors 
that were made* The negative results do not allow any conclusions 






1^8 



to be made on the relative value of tutorial versus programed in- 
struction for remedial work in English composition. 

Project 2 ; In this evaluation study two classes of freshmen 
participated for a period of two weeks. Essays were written for 
both the pre and post tests and the score was the number of sen- 
tences written per type of error made. 



Group 




II 


test 


Post test 


Gain 


Program 


Mean 




6.4 


10.4 


4 . 0 




Ho. in 


Gp 


21 






Control 


Mean 




4.9 


5.6 


. 7 


(regular 










instruction) 


Ho . in 


Gp 


14 


• 





The two groups averaged between 15 and 20 sentences on both 
the pre and post test essays. The programed group showed consid- 
erable gain in eliminating different types of errors in their 
essays. The control group using regular instruction made only a 
slight gain. The results indicate that the program can make a 
significant contribution in eliminating types of grammatical errors 
in freshman English classes when used as a primary teaching 
method for a short unit of material. 

c. Instructor Evaluation 

Professor Chitted indicated that she had used programed 
material before. In evaluating the programed material she used* ■ 
Professor Whltted said the following: ' 

^^P*^^^^cnts in Wrxting by Smith and Staplef ord is $ as X havjB 
indicated, the best book of programed material dealing with the 
sentence that I have seen. However, it seems to me to have two 
major flaws. 

1) Levels of difficulty 



1 

The book ranges from very simple to extremely complex 
sentence problems. Probably the authors intended to start 
with something very simple (agreement of subject and verb) 
to give the user of the book a feeling of confidence. 

With collage students it seemed to give a false sense of 
the book*s being a review of elementary material. 

Chapters 2 and 5 are the two next easiest chapters, and 
they lend themselves to programing more than the others, 
although 3 and 4 can be explained more easily than 6 and 
8, which are by far the hardest to get across through the 



JLaji. 






€ 



mediuin of programing* 
prompt the comment of 



It is chapters like these that 
one boy, "You can*t ask a book ‘ 



# 



explain something that you don*t understand • 

2) Omission of important material 

The most common and most critical error in sentence con- 
struction made by poor writers is the comma fault. No- 
where in the programed book is this structure discussed. 
By Implication it is taken care of in the chapters deal- 
<ing with subordination of sentence elements, but- student 
needing to use the material at all are unable to make 
such remote connections. Consequently some very bad 
errors in the papers could not be pinpointed in the text 
though they related to either subordination or 
coordination." 

i 

, Th6 InstiPiictoi? considcped that *th6 cbB*t€n*t covet*6d by tha 

program justified the amount of time required by the student. The 
program saved the instructor time but she was uncertain as to 
whether it was justified in terms of its cost. For her individual 
students and for her tutorial program. Professor Whitted Indicated 
that she was more negative than positive toward programing. For thf 
group experiment in project number two, however, she wss more 
positive than negative* 

Professor Whitted commented as follows: 

. * j 

When I looked over the responses of the students who worked 
with the programed book, I was struck by the fact that the 
better writers tended to like the book and to think that the 
had gained good review from studying it; the poorer writers 
were the ones who complained about the frustrating order of 
the book and who seemed to-be convinced tliflit the whole book 
was a waste of time. These responses seem to bear out the 
point that the poorer writer is frustrated by having to work 
independently with material that is not orderly and that -he 
does not always recognize his own weaknesses. As in the case 
of the^ remedial student, he often needs the support of a 

human being as well as the requirement for meeting frequent 
dd8idHn68 • 

However, programed material seems to me to lend itself to use 
in a class of reasonably good freshmen students much more 
than of ^students who have problems in composing -- older stu- 
dents with more deepseated problems, those who are remedial 
cases. But I am sure also that certain kinds of people of 
any age respond to the programed material better than others. 
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13. Professor Franois Williams who teaches Organic Chemistry 
at Antioch College used Professor Theodor Benfey*s prograri Syste- 
?matic Warning of Aliphatic Compounds . 



Method 



The program wds used as a completely independent study. 
Williams did not lecture at all on the naming of these compounds 
(during the course. The students were required to learn the nomen 
Iclature complete from the program. 



The experimental group consisted of 26 students who took 

A « A _ 



jWilliams* course in organic chemistry during the summer. The con- 



jtrol group WES a group that took the same course in chemistry in 
^he idll. 






For the experimental group, Williams gave out pre tests before 
the course began. He also handed out the total program consisting 
of six sections. Ho referred students to programf for all nomen** 
blature and assigned nomenclature problems from tl>e book, posting 
pnswers on the bpard. No help was given to the c^liass on the nomen- 
clature problems. After this procedure he gave post test. 



I The control group consisting of 18 students was involved in 
the Fall quarter. They were given the pre-test , »'|iid unspeqif led ^ 
number of lectures during the quarter and then tj^p^post-tes^ ”” 



b. Results 



I The experimental group had a mean gain score of 44.5; the con- 

jtrol group had a mean gain score of 41.9. Although there was a 
significant amount of learning that took place in each group, no 
significant difference was found between the mean gain scores of th€ 

two groups. The programed materials taught as well as the traditloi*' 
hi lecture method. * 



Student Evaluation 



i 



K'* * 



This was the first experience with programed material for 23 
of the 28 students using the Benfey program. Eighty-two percent re- 
acted favorably to the program. Without an instructor the students 
preferred the program to a typical text, but a majority preferred 
regular lectures to the program. An overwhelming ninty-two percent 
indicated that the time spent on the program was worth it consider- 
ing how much they learned. _.They Celt it speeded up their learning 

but tended to be quite mechanical. 



The group felt that the best way to. utilize programed material 
It the college level was as a supplement to textbooks and lectures. 
Jeventy one percent felt that the program had no appreciable effect 
on their attitude toward the subject matter. 
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d . Instructor Evaluation 

Williams had not used programed material before. He used the 
program as a primary teaching device in place of textbook and lec- 
tures. He described his attitude toward programed material as be- 
ing favorably disposed to it. Regarding the results of the program 
he said the following: "I got excellent results on this material 

which requires rote systematic study. It also relieved me of the 
necessity of covering this in lectures. I don*t like to lecture on 
it and it is not well received when I do.” 

"My subjective opinion is that the students did well, learn- 
ing more than I would otherwise have required in no more study time. 

I saved lecture time." ^ 

In summary his overall reaction to the program he used was 
ivery positive and he would like to use similar material again. He 
,said, "From material of this sort, rote learning of very systematic 
subject matter, lt*s the only way. I think we should attempt to 
push as far as possible into unsystematic subjects but be ready to 
.stop wherever it falls." 

I 

j 14. Professor Robert Wilson, Mathematics Department of Ohio 

•Wesleyan Unlvers|lty, agreed to conduct a research project comparing 
two commercial programs covering elementary functions in mathematics. 
|He used the following programs. TMI-Grolier Algebra Refresher; 

•Wiley* Programed Beginning Algebra ; TEMAC Analytic. Trigonometry . 

{ a . Method ^ 

^ The specific programs were used on two grouj^s of 25 students 

jeach. The students selected for the study were those who had to 
jmeet a mathematics' proficiency requirement and did not wish to ta|s| 
ja form of college-level mathematics course. The students were to 
matched on a proficiency test in mathematics as well as a check on 
(high school mathematic grades. All work was done in a room provided 
Tutorial help was available and no post test was given. Evaluation 
was based upon the judgment of the tutors, student evaluation 
questionnaires and the instructor's program evaluation. 

b. Student Evaluation 

Nine students completed the attitude questionnaire and for two 
thirds of these students, this was their first experience wdth pro- 
gramed materials.. 

All nine of the students Indicated that it was a positive 
learning experience* Seven of the nine said they would prefer to 
Learn the materials on their own by a program rather than by regular 
textbook* 
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Eight out of nine claimed that what they learned from the pro' 
gram justified the time spent on it. 



When asked to indicate the best ways to use programed instruc 
tion in oollegOi the three responses checked most often were as 
follows: 



No. 



Response 



8 

6 

5 



For remedial work when the student falls behind 
To bring students up to starting level in the cours 
As a supplement to textbooks and lectures 



When asked, "How has the use of programed material affected 
your attitude toward the subject matter of the course?", four re- 
sponded favorably and five said it had no appreciable effect. No 
one responded negatively. 



The summary of the responses on the student attitude question 
naires indicate a rather strong favorable reaction to programed in- 
istruction as used in the context of this study. 



Professor Wilson substantiated this conclusion with the 
ollowing statement. "The students who are partiqipating in this 
^ rogram and who have taken the programed learning develop those 

f “kills which they did not have on entering Ohio We||eyan unanimously 
ave very high praise for this program. Several hfye told me that " 
l«ven though they may be able to pass a competency lj|aminatlon beforf 
they have completed all of the programr^d material, they intend to 
pome back and complete the material for they feel they are learning 
^ great deal that they missed in their secondary mathematics." 



f 



Instructor Evaluation 



e 



The Instructor indicated that he had not used programed mater- 
ial 'before. The programs used in this evaluation were used for re- 
nedial work when the student was falling behind (trigonometry); to 
)ring students to a starting level of the course (algebra and trig- 
>nometry) and to teach material that the Instructor did not want to 
’.over in class (trigonometry). 



His attitude before beginning the evaluation study was enthusi 
istlc (for remedial emphasis). He said, "I was Interested in it but 
lot sure of the results." 



Continuing to describe his reactions to evaluation of program- 
sd material Professor Wilson said, "The programs used were all good 
*ind accomplished the purposes for which they were adopted. The 
niey program in algebra appeared to be slightly better than the TMI 
Srowlier program on the basis of the first results. 
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His specific reactions to programed material were as follows . 
He said that the program actually covered mot*e than what would be 
covered in a remedial course. Ho found no errors in the content o:? 
the program nor did ho observe any errors in the technical aspects 
of the program. He indicated that the importance of the material 
covered by the program justified the amount of time required by 
the students and that it saved his time as an instructor. The pro* 
gram also was justified in terms of its cost. 



He know of no better or similar programed materials and in- 
dicated that he preferred the programed material to both textbook 
and lectures for remedial work. In summary • his attitude toward 
using programed material the way he did was very t>oEitlve. 



DISCUSSION 

A. Effectiveness of the Programs 

The sample of commercial programs evaluated was in no way 
intended to be comprehensive or exhaustive. The programs were use< 
under a variety of conditions believed to be beneficial to their 
students by the instructors involved. Nevertheless, some inter- 
esting observations can be made and used by interested faculty 
members to guide their selection and use of these programs.^ 

First, in the natural sciences, two biology programs were 
tested. Kormondy found that his own commercially published pro- 

Introduction to Genetics , produced better results than either 
comparable material in a regular textbook or presenting the mater- 
lectures. Dillery was Interested in detelpmining if studentr 
would utilize programed material when it was made available to 
them. The biology program used in the study was>. Cells: Their 

Structure a nd Function . As might be expected, the poorer students 
did not read the program, even if it was assigned. The average 
and above students used the program more if the material was ex- 
plicitly assigned rather than merely mentioned as a supplementary 
resource. 

Two chemistry programs were tested. Reinheimer was quite 
pleased with Barrow* s Programed Supplement for General Chemistry 
used as a review for unit examinations in Freshmen Chemistry. 
Williams evaluated flenfey's program. Systematic Naming of Aliphatic 
Compounds . His attitude tbward the program was vei^~positive , ’ 

claiming that in his subjective opinion the students did well. 

"They learned more than I would otherwise have required in no more 
study time. And I saved lecture time." 



In mathematics, Smyth evaluated the TENAC program. Analytic 
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Trigonometry. The study lasted for an entire semester. The results} 
indicate that students learned as much from the program as from lec •• 
tures supplemented by discussions. However, both the Instructor and 
the students expressed negative reactions to the program. 

Wilson evaluated three mathematics programs for use with 
college freshmen. His subjective evaluation was that all three pro* 
grams were beneficial for remedial work or to teach material not 
covered in class. He Tlslalmed that the students were enthusiastic 
about the programs also. 

Wetmore compared two commercial programed texts in educational, 
statistics. Ho believed that Gorow*s text. Statistical Measures , 
was superior to Bradley and McClelland's Basic Statistical Concepts 
However, the gain scores based on pre and post tests did not reveal 
significantly different group mean scores. 

Hackstaff evaluated two programs in logic apd found both to 
bo unsatisfactory. He stated that the text. Class Logic , was in 
most ways superior to Basic Logic but neither was a good substitute 
for a general technical text in sym' olic logic. 

Whitted evaluated the program. Effective Wri^ting . but Smith 
and Stapleford under two diverse conditions. As § remedial teach- 
ing device to overcome deficiencies in English copposltlon , the 
program was unsatisfactory. However, with regular ; freshman English 
classes the program p/oved to be very helpful. 

t 

Weiss compared two commercial programs in speech. The results 
indicated that both Lehman's text. Parliamentary Procedure, and 6ra3 
and Rea's text, with the same title, resulted in significant learn- 
ing by college students. 

London and Van Liere both evaluated the well known text. 
Analysis of Behavior by Holland and Skinner. London found that it 
made no difference in achievement whether the students wrote answers 
lin the blanks provided in the program or read the material in prose 
jstyle with no overt response necessary* The group which was re- 
Ijuired only to read the material finished the program much faster, 
and consequently this method of presentation proved to be more 
efficient in terms of achievement than requiring students to fill 
in blanks. 

Van Liere used the same text as a primary teaching instrument 
ifith one group and to supplement lectures with another group* Both 
these experimental groups scored significantly higher on an achieve- 
ment test covering the material than a control group which heard 
lectures only. The students were positive in their attitude toward 
;he program • but the instructor was quite negative in his reaction 
to the value of the programed material. 

. lil.fi 
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, Finally, Charles and Rodriquez evaluated the TEMAC program, 

-French Phonetics » Both instructors judged that the program alone 

teaching French pronunciation to beginners. 
Nevertheless ,’^tua¥nts did make some progress in pronunciation from 
jusing the program. 

I In summary, the evaluatidiT commercial programs did not 

^change instructor attitudes toward' programed instruction. Both 
jfavorable and unfavorable attitudes were retained. In two instancen 
jthe instructors acknowledged the significant achievement made by 
their students using the programs, yet they remained negative. towar< 
programed instruction. 

i 

t The results on achievement tests covering the material in the 

.■programs were too tentative to be used as a basis for judging tlse 
jvalue of any particular commercial program. The Normandy and London 
jstudies were the most carefully designed and executed. Therefore » 

^n these two studies considerable weight can be given to the objec- 
|tive results which are reported. *For this reason, these two studien 
are reported in much greater detail than the others. 



p • Effectiveness of Evaluation Procedures 

I 

\ Each instructor was asked to judge the eff eclfiveness of the 

procedures used to evaluate the program and then qffer suggestions, 
ffor improving the study. A large majority of the ^valuators be- 
lieved^ that the studies accomplished what they had"' hoped would be 
accomplished. Only minor changes were suggested to improve the 
studies. 

I 

I 

j 

Q Ag, notAc oahlo wea kne s s in many of the studies was the tend- 
ency to gather much more data than could possibly be tabulated and 
analysed. If evaluation studies of this nature are to be of value, 
the scope of each study must be clearly delineated and kept from 
becoming hopelessly broad or complex. 

CONCLUSIONS 

The evaluation of commercial programs provided first-hand 
experience to faculty members and students in the GLCA colleges witl 
what was currently available on the market. 

For a large majority of the students, this was their first ex- 
perience with programed materials and the reaction in general was 
favorable. Reactions of faculty members Nho evaluated were mixed, 
ibout half favoring programed instruction and half disfavoring it. 
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Mo pattern was dlaoerniblo in the teachers' reactions to the 
materials nor in the achievement of the students using them* 

The studies provided the 6LCA staff with an opportunity to 
try out a wide variety of program evaluation procedures* Although 
the results were indeterminate with respect to any overall con- 
clusions about the value of programed materials at the college 
level, they were beneficial in providing guidelines to the de- 
signing of much better controlled studies of the GLCA-produced 
programs w*jioh were evaluated the following year, and the results 
of which are reported in the next chapter* 
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Chapter 6 

EVALUATION OF GLCA-PRODUCED PROGRAMS* 



INTRODUCTION 



In Chapter 2 of the report the administration of the evalua- 
tion project was described. Chapter 4 reviewed related research. 
Chapter S reported the results of the evaluation of commercial pro- 
grams and showed the need for more tightly controlled experimental 
designs. The experiment reported in this chapter was designed to 
gain evidence concerning three major questions: First, do pro- 

gramed materials teach? Secondly, do they teach as well as other 
methods of instruction? Thirdly, how can programed materials be 
most effectively used in college teaching? 



Four specific questions were selected to serve as a focus for 
the evaluation of programed instruction at the college level. Thes 
questions were chosen because they seemed to be peiSttinent to the 
jinterests and concerns that GLCA faculty members |i(|id about 
{grarntd instruqtii<jp» Th« questions are: 1) Do 



pi)Q|ram8 



pro- 
tsaoh as 



ttffeotiveiy as teKthoeks or leotures oovering the asms material? 



2) Nhat olas8r4p|i activities such as question aq4t answer dlscussiof s 
jr lectures fol^qiKlng the student's use of a prog|pf|(p imost effectlve4 
ly capitalize op" what students have learned in thl^lsrogram? 3) Is 
the program more effective as an instrument for thO^ acquisition of 

f nowledge or as a device for reviewing information acquired earlier 
y some other means? 4) What effect do various motivational device! 
lhavei on learning by programed materials versus other methods? 



methods 



Our concern was not only with learning as measured by tests 
aut also in student attitude toward and instructor evaluation of th< 
ise of programed materials in comparison With other methods of in- 
struction. Therefore, each student and instructor participating in 
the program was asked to express his opinions about various aspects 
jf programing and other methods of instruction on an attitude 
luestionnaire. 



Rescript ion of the Sample 



The twenty— four studies included in the experiment were con** 
•luctdd on the campuses of nine relatively small liberal arts 



* Dr. Donald Beane is the principal author, of this chapter. 











colleges and universities located in the Middle West. Eight of 
these college and universities are members of the Great Lakes 
Colleges Association: Albion College^ Denison Universlty« DePauw 

University, Hope College, Earlham College, Kenyon College, Ohio 
Wesleyan University, The College of Wooster. Illinois Wesleyan 
University, Bloomington, Illinois also participated in the evalua- 
tion project. 



The following twenty six professors participated in the eval- 
uation of the programs: Daniel Anderson, Brad Angell, Philosophy 

Department, Ohio Wesleyan University; Thomas Boyle, English. Depart- 
ment, Albion College; T.R. Burkett, English Department, Denison 
University; Philip Church, English Department ^ Kenyon College; 
Robert DeHaan, Education Department, Hope College; Robert Johnson, 
English Department, DePauw University; William Judd, English De- 
partment, Ohio Wesleyan Unlvei ^ity; Mrs. Sue McNaghton, Department 
of Government, Denison University; Ray Mizer, English Department, 
DePauw University; Roy Morey, Department of Government, Denison 
University; Wendall Patton, Zoology Department, Ol;io Wesleyan 
University; Morton Schagrin, Department of History of Sciences, 

I Denison University; Lee Scott, Philosophy and Religion Department, 
i Denison University; William Richard Stegner, Religion Department, 
j' Illinois Wesleyan University; William 5tephenson|*'B4ology Pspart* 

! iment, Kuplhtm Jei^vy Itene, Religion Depifitment , Illinois 

jWesleyan Univerlity; Jerome Tovo, Philosophy Dep^f^tment, College of 
i Wooster; Philip Van Eyl, Psychology Department, Hope College; 
"elvin Vulgamor|i| Religion Department, Ohio Wesl^^yan University; 
harles Weis, Ei^||lish Department, Ohio Wesleyan i^piyersity; William 
estbrook, Ecotiomics Department, Denison University; Louis Wilcox, 
iology Department, Earlham College; Vannie Wilson, Biology Depart- 
ent, Denison University; Fred Wirt, Department of Government, 
enison University; Frank Yow, Biology Department, Kenyon College. 

A total of 1220 students from the nine colleges and univer- 
ities participated in the project. 




progra ms t o be E valuated 

T" imn-'rm >i ~ mm— — — — — 

! The Great Lakes Colleges Association had authorized during the 

Bummer of 1964 the production of twenty-four programs by professors 
ftn the 6LCA institutions. These were field tested in the Fall of 
1964 and revised. In the Spring of 1965, five of the bert programs 
gere selected for the systematic evaluation described herein. These 
five programs were in Engli'sh Poetry, Religion, and Logic, Biochem- 
istry, and Political Science. The subject matter covered by the 
programs represents the three major divisions in the liberal arts 
surricttlum: humanitiee, natural sciences and social sciences. 
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Dotailed descriptions of and sample pages from the* five pro- 
grams that were selected for inten,sive evaluation are found in 
Chapter 3* The titles and authors are given below: 



POETRY: METHOD AMD MEANING , 

James W. Cook, Department of English 
Albion College, Albion, Michigan 

LANGUAGE OF LOGIC, 

Morton Schagrin, Department of History of Sciences, 
Denison University, Granville, Ohio 

e 

BIOCHEMISTRY FOR BIOLOGISTS, 

William K. Stephenson, Department of Biology, 
Earlham College, Richmond, Indiana 

STUDIES IN THE GOSPELS ‘ 

Robert Montgomery, Department of Religion, 

Ohio ffesleyan University, Delaware, Ohio 

AM IlftiQDUCTIOM TO SYSTEMATIC ANALYSIS POLITICAL 

' ' . SCIENCE 

Lola Palakeudaa, Dapartmant of Political Soianoe 
Antioch College, Yellow Springs, Ohio ' • 



Description of Evaluation Designs 



1 . 



2 . 



3. 



4. 



5. 



E^valuation^Design #1 . The following design was used to 
answer the first question : Do programs teach as effectively as 

textbooks and/or lectures covering the same material?' Below is a 
schematic diagram of the procedure for evaluating programs in com 
•^arisen with other methods of instruction. 



Table 1 ~ Summary of Evaluation Design itl 



Sroup Step 1 


Step 2 


Step 3 


Step 4 


Step 5 


K Pre-test 

j Form A 


Programed 

Material 


Post-test 
Form A-1 


Discussion**-*^ 
1-3 periods 


Final Test 
Form A 


p Pre-test 

1 Form A 


Textbook 

Material 


Post-test 
Form A-1 


Discussion^** 
1-3 periods 


Final test 
Form A 


p Pre-test 
Form A 


Expository 

Lecture 

Material 


Post-test 
Form A-1 

* 


Discussionl4 
1-3 periods 


Final test 
Form A 



immuxi- -t tmrnm 



fSV> 



' ' 

The term "Discuasion" used throughout this chapter means a 
student originated quest ionrsnd-answer session. 
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All groups were given the same test in Step It '*** alternate 
form of that test in Step 3 and the original test form again in 
Step 5. The students were asked to write 5 questions concerning 
the material covered in Step 2$ snd these questions formed the basl 
for the class discussions in Step 4* These class discussions were 
taped* The lectures were also taped to insure that the lectures 
covered the same material as that presented in the program and^text** 
books used in the experiment. In most of the studies under this 
designt, two groups were compared rather than three. 



Evaluation Design #2 . The 
swer the second question* Hhat 
cussions or lectures, following 
effectively capitalize on what s 
gram? This design is basically 
Group A Design #1* Below is a s 
sign for evaluating programs in 
that follow thpir uses 



following design was used to an~ 
classroom activities, such as dis- 
the students* use of a program most 
tudents have learned in the pro- 
an expansion of procedures used on 
chematic diagram of the basic de- 
terms of the educational activities 



Table 2 - Summary of Evaluation Design #2 



iGroup Step 1 


^ Step 2 


Step 3 


Step 4 


Step 5 




A 


Pre-test 
Form A 


Programed 

Material 


Post-test 
Form A-1 


Discussion ' . 


Final Test 
Form A 




B 


Pre-test 


Programed 


Post-test 


Review 


Final Test 






Form A 


Material 


Form A-1 


Lectures 


Form A 




C 


Pre-test 


Programed 


Post-test 


Combination 


Final Test 






Form A 


Material 


Form A-1 


of methods'. 


F oriTi A 





I The discussion for Group A was student-cent-^red with the in- 

structor taking no initiative to raise questions. In contrast to 
this. Group B received pre-structured lectures and any question and 
answer period following the lectures was structured by the professoi 



Evaluation Design #3 . The following design was used to an- 
swer the third questions Is the pregram more effective as an in- 
strument for the acquisition of knowledge or as a device for re- 
viewing information acquired by some other means? Below is a 
schematic diagram of the procedure for evaluating programs in terms 
of when they are used: 
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Table 3 - Sumimary of Design #3 



Group 


Step §i 


Step J2 


Step #3 


Step 


#4 


A 


Pretest 


Programed 


Supplementary 


Post 


test 




Form A 


Material 


Lectures 


Form 


A-1 


B 


Pretest 


Supplementary 


Programed 


Post 


test 




Form A 


Lectures 


Material 


Form 


A-1 * 

• 



The same programed material was given to both groups. Al- 
so both groups received the same number of lectures with approxi- 
mately the same content. 

Evaluation Design #4 . The following design was used to ans- 
wer the fourth questions What effect on learning have various mo- 
tivational devices such as volunteering vs non-volunteering and 
being graded vs. working for no credit? j 

In two stu4i*> * design sifnilar to Design was used but the 
students voluntiB fifed to participate in the experiiiient on their own 
time outside of' class. They volunteered to work pp a supplementary 
unit related to course work but were not in, what manner 

the material wc^|i|;d be presented. Then half were'‘*g'|yen' the program 
and half received lectures over the same content. 



'Table 4 - Summary of Design #4 





Group 


Step §1 


Step #2 


Step 


#3 


A 


Pre test 
Form A 


Programed 

Materials 


Post 

Form 


t|St 


B 


Pre test 
Form A 


Lectures 


Post 

Form 


test 

Al 





In the third study under Design #4 both volunteers and non- 
volunteers were used. Half of each group were told they would re- 
ceive .extra credit for the unit if the test scores on the unit 
would help their course grade. The second half were told the grade 
of this unit of work would count toward the final course grade re- 
gardless of how well or poorly they did. All students in this stud; 
used the programed materials exclusively. 
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Table 


5 - Summary of Study 


^3, 






Evaluation Design #4 








Extra Credit 






Volunteer Group 


1 


2 




Mon Volunteer Group 


3 


4 

• 





Descpiption of Tests used in the Evaluation Stud 



les 



The tests used to evaluate student learning were written by 
the authors of the programs with the exception of the political 
science tests, which were written by Fred Wirt, one of the evalua- 
tors of the program. The tests were 40-50 minutes in length with a 
format of questions 50 percent objective and 50 percent essay in 
nature. Two forms of each test, parallel in format and content, 
ware written. • 







't 



The preliminary forms of the tests were edi'j^ed by subject 
matter specialipts in the disciplines concerned and then revised 
prior to use ip the evaluation studies. (A copyrqf one form of each 
test is included in Appendices E-1, E-2, E-3, E-|!| E-5) • 

■iL ^ 

A summary of individual studies, programs , ^ and designs appearii 
on the next page in matrix form. The professor(s) conducting each 
of the twenty-four studies and the college or university where the 
evaluation study took place are also indicated on the chart. 
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Table 6 - Summary of 1964 GLCA Programs to be Evaluated 




Mizer and Anderson 
Johnson OWU 
DePauw 



Stephenson 
and Wilcox 
Sarlhaito 



Vulgamore 

OWU 



Wirt 

Denison 



Design 

n 



Church, Angell 
Kenyon OWU 



Stegner 

Ills Wese U» 



Judd and Van Eyl Yow 

Hope Kenyon 

OWU 



Design 



Burkett DeHaan 

Dtnlwon Hopt 



I 



Stephenson 
and Wileox 
Earlham 



Scott 

Dtniaon 



McNaghten 

Dtnifon 



Desl.gn 
#3 



Boyle 



Schagrln Patton 
Albion . penison , OWU 



Stegnen and Morey 
Stoner: Denison 

^I13,.Wp|.U. J 



Design 
#4 



To VO 
Wooster 



Wilson 

Denison 



Westbrook 

Denison 



T 



RESULTS 



^jnmary of Composite Analysis of Learning 



^ summarises the analysis of learning. In all 24 r+uA- 
icent*'L^ZA'*"* learning by program and other methods was signif- 
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parentheses means that in 23 of the 
studies the obtained results, le, the size of the increast 
In post test scores compared with pre teat scores, would*’ 
have^occured by chance (p) less thSn once irHSou^^nf 
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In four of the studies under Design #1 the program was com- 
pared with text materiala In one case the program surpassed the 
textbook, while In the other three studies, there were no signifi- 
cant differences In amount of learning between programed materials 
and text materials. 



When the programed materials were compared with lectures, the 
programed materials resulted In significantly more learning In threis 
Judies. In ^oiir studies there was no difference In the amount of ^ 
learning , and one case (Mlzer-Johnson) the data were Inconclusive. 



Under Design ^2 the four studies where valid comparisons 
could be made Indicated no differences In learning due to the 
methods used following use of the program. This would lead one to 
conclude that he could follow the use of these programs by either 
discussion or lectures and obtain similar results. 



In Design the evidence indicates that students learn as 
well whether the program is used before or after the supplementary 
lectures. ^ 



In Design ||4 students volunteering to study the material 
oovered in the pregrem did no better nor woree o^ the unit then 
those etudents ppo were required to learn the material as a regular 
part of the cot|]p|ie. 



<* 



The results of each study by program that w#e evaluated and 
design are presented on the following chart: 
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Table 7 
Design 



Summary of Results of Analysis of Variance by Program and 






• * • «S* . 



Design 

n 



Dgsign 






Design 

n 



Design 
#4 




Lng - Learning as measured by differences between pre and 
post test scores 
P - Programed materials"“ — - 
L - Lectures 

Eg»» P>L means that program 
was ^ superior to lecture* 



T - Text materials 
D - Discussion 
V - Volunteers 



Poetry 


Logic 


Biochemistry 


Religion 


Pol* Science 


Hizer** 

Johnson 

Lng*** 

P vs* L^ 


Anderson 

Lng *** 

P = T , 


Stephenson- 
Wilcox 
Lng *** 
p>L ** 
pjrT ** 


Vulgamore 
Lng *** 
* 


Wirt 
Lng *** 
•P a L 
• 


Church 
Lng *** 
P = L 


Angell 
Lng *** 

P US Y 


Yow 

Lng *** 
^ P>-L *** 


Stegner 
Lng *** 
P = L 




Judd'-Weis 
Lng ** 

P a L 


Van Eyl 
Lng ***‘ 

P a T 








Burkett 

Lng *** 

P + D«PfL 


DeHaan 

Lng *** 

P + 0«P+L 


Stephenson- 
Wilcox 
Lng ^^*^ 

P + D • P+L 
=P+D (Mixed) 


Scott McNaghten 

Lng Lng 

* 5^ 

P + D vs P+ P+D = P+L 
L^ 


Boyle 
Lng *** 
tPfLsL+P 


Schagrin 
Lns *** 

P-f^^L+P 


Patton 
Lng *** 

P + L-L+P 


Stegn'^l^r-Stone Morey 
Lng **^ Lng *** 

P + L=|+P P+L=L + P 


# ■ 


Tovo 
Lng *** 
P = L 


Wilson 
Lng *** 

P a L 




Westbrook 
Lng *** 

V = MV 



MV - Nonvolunteers 

A •" significant differences 

obtained ru 

W W ^ e *1 ** f M A M M ft* ftftft ^ Jk mm. ^ aA mm m. m ^ ftft ■« mm 



A a"" • T — ojT xne ooxainea, 

Trr' °°°"*^*^ hyp-hanr.. x.nft mean* Jbflt tW 

* 5 100} <.01 Cprobibllit' 



ian 1 In 100}<.0(il - l«»s than 1 in looo. 



data 46t odtaclaaiv* aino* gvoupa hdt odnpavabla du* to aim 
fleabt diffavaneaa in neanfog pna-teat aeopa*. 



t A«»i’ ;IWI ltd fe'WHUir » 
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Attitudes toward Programed Instruction 



Student Attit udes; Summary of All Studies Combined . One of 
our major areas of evaluation concerned student attitudes toward 
programed instruction. Did the students think that learning by 
programed instruction was a positive learning experience, regardles 
of whether tests indicated they learned or not? 



He investigated student attitude toward methods of instructio 
from four separate points of view. 



n 



1. General impression of the learning experience. 

2. The pace or rate at which the information was presented. 



3. The sense of direction which the method of instruction 
conveyed. 



4. The ease or difficulty with which the material presented 
couxd be reviewed. 



For the exact wording of the questions and the five responses 
to each, refer to Questions 4, 5, 6, and 7 of the teaching-Method 
Evaluation Questionnaire in Appendix E-6e 



Scoring the responses from 1 to 5, we calculated mean ranks 
for each method of instruction used in the evaluatlone The mean 
ranks and number of students responding to each question are nre- 
sented in Table 1, Appendix E-7, ^ 



8 



Examination of this table roveals that students thought all 
four methods of Instruction provided a more positive than negative 
learning experience, the pace was about right, and the students had 
% ?!”!! direction most of the time. In general, the students 

1 review was fairly easy for programed materials, textbooks 

and lectures, while review of material presented in class dis- 
cussions was fairly difficult. 



A statistical analysis compared attitudes toward programed 
materials with those toward other methods of instruction. The two- 
tailed t-ratlo * was used based on correlated observations to test 
the null hypothesis of no differences. The tests revealed that 
students had a significantly more favorable attitude toward pro- 
gramed than discussion as a positive learning experience, 

t(505) -4 5.47, pi^.OOl. Students felt the pace of programed 



A 



M 









i 






The two tailed t-ratlo is used when the investigator is not 
^“tarested in which group is superior, only if there is a ^ 
Significant difference in the means of the two groups. Seh 
Appendix E-9 for the formulas used in computing the t-rati>s. 
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materials was better than learning by class discussion t (477) = ' 

3.80, .001 and the sense of direction was significantly better 

^(486) ss 4.70, p<*.001. Review of programed materials was also 
judged to be significantly easier than review of class discussions, ‘ 
t (443) = 12.46, p< .001. 

The only significant differences between programed instruct 
tlon and lectures was in the sense of direction. Students believed 
that lectures did a better job in this regard than the programed 
materials, t (325) = 2.90, p<.001. 

0 • 

P rofessor Attitudes; Summary of All Studies Combined . The 
college professors who conducted the program evaluation studies 
filled out an attitude questionnaire similar to the student ques- 
tionnaire. The questionnaire was completed at the end of each ex- 
periment. A copy of the questionnaire is included in Appendix -B. 

We were interested particularly in whether participation in 
a programed material evaluation study would change any professor’s 
attitude toward programed instruction. Questions #7 and #31 were 
designed to provide an indication ofv,^ange in attitudes. ^ 
sununory of the number of professors oh^osinff each response to these 
two questions are presented below* 



Q#7 Describe your attitude toward programed instruction prlop 
to evaluation of the material for GLCA* ^ 



4 Enthusiastic 
12 Favorably disposed 
10 Uncertain 
2 Unfavorably disposed 
0 Strongly disagree 

Q#31 In summary, what is your overall reaction at this time 
to using programed material in tirin- course? 

14 Very positive, would like to use similar material again 

15 More positive than negative reaction to it 
0 Indifferent to using it 

0 More negative than positive reaction to it 
0 Very poor, would object to using it again. 




« 
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1 


Tho change to a more favorable attitude toward programed 
materials was very striking after personal experience with a good 
program* 


II 


1 

ll 


In evaluating . the pace of the program^ the professors res- 
ponded as follows: 




» 

1 


3 too fast 






1 Too slow 


• 


1 


6 Uneven 

• 




D 

\ 


18 About right 




^ \ 

e 

? 

... ^ 1 


The pattern of responses closely paralleled those of the stu- 
dents ^ who also felt the pace of the programed materials was about 
right* 


• f 


i 

! 
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Question #18 concerned the sense of directibn which the pro- 
gram provided*. The professoz^s responded as follOffs: 

• i* * 




I I 

! 

1 

6 

t ^ 


Doss ths studsnt gst a ssnss of dlrsotion tfhsrs hs is going 
from the program? ' 




1 

1 • 


19 yes 




0 ‘. 

i ? ‘ 


3 no 




1 

; - If. 

; !.. 


5 'at times 




|- 

11 

i) 

i 
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One criticism which is frequently made about programed 
materials is that they are of little use for review purposes* Ques 
tion #19 asked for professor attitudes in this area* The responses 
are given below: 


mm 


I ■ 


Can the student easily review the programed materlalV 




n 


21 yes 




* \} 


4 no 




i ^ 

1 -: 
8, 


1 at times 




1 

t 


The following three questions call for evaluation of the pro- 
gram as a practical instrument fo 7 £* learning* 




‘1 . ^ ., 
.. i ■ 

1 

' r 

'•I' 


Question #13* In your opinion, does the program actually 
teach what it claims to teach to the students for. whom it is pre- 
sumably designed? 


\ 


^ 'I-'- 

; eric y : 


' ' ' ' ^ "■ A ^ ' f 

y . ' ''' ! 


- . " ihr:. 
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4 partly 

Question §21 » Does the importance and amount of content 
covered justify the amount of time required of the student? 

20 yes 

1 no 

7 uncertain 

Question #22* Does the program save the instructor time? 

23 yes 

3 no 

2 uncertain 

In conclusion^ the professors believed the programed mater- 
ials taught the students something worthwhile in 4 reasonable a- 
mount of time with the added bonus that the professor saved valu- 
able time. ^ 



Analysis of Students* Attitudes Toward Specific Programs 

Poetry Program . The mean ranks of all students participating 
in one of the studies evaluating the poetry program are given in 
Table 2| Appendix E-7. The difficulty of reviewing material cover- 
ed in class discussion is the only deviation from the general 
pattern of responses to all four methods of instruction. The mean 
rank of 2.53 for review by discussion indicates that students per- 
ceive discussion as being more difficult to review. 

The analysis by t-test comparing programed instruction with 
lecture and programed instruction with discussion revealed the 
following significant differences in mean ranks. 

Students viewed lectures as a more positive learning exper- 
ience than programed material. ^(74) * 2.48, p<.05. 

Review was easier by programed materials than by discussion. 
t(80) - 5.16, p< .001. 

: . fcoRic Program , the students expressed similar attitudes 




H 



t 



itoward all four methods of instruction in regard to the logic pro- 
gram* The one exception was again in the area of review where clast 
discussion was judged by students to be the most difficult to re- 
view* iKee Table 3 Appendix E-7*) 



The analysis by t-tost revealed the students felt that the 
program materials were significantly better than class discussion at 
a worthwhile learning experience ^(88) s 2*54, p<*05; in sense of 
direction, ;t(74) = 2*01, p^*05$ ¥nd in ease of review, t(62) ■ 

2*37, • o¥* 



In comparing the programed materials with lectures, the stu- 
dents preferred the programed materials as a positive learning ex- 
erience, ^(62) = 2*94, p< *01* However, the pace of the programed 
iaterials was too slow compared to the lectures t(64) = 3*14, 

;>< * 01* 






Biochemistry Program * The mean ranks of studept responses (see 
Fable 4 Appendix E-7) indicate they considered the programed mater- 
ials to be more positive than negative as a learning experience and 
;hat the pace was about right most of the time* The program pro- 
ided a sense of direction to the student and he found it easy to 
‘svltw. 

In comparing the program with lecture and with class discussion, 
jthe following results emerge ? The students preferred the programed 
oaterials over discussion in three areas evaluate4;in this study* 

a* positive learning experience it(175) * 9,58, p<*001 

b. sense of direction lt(172) = 4*37, p^*001 

c* ease of review 1t(153) ssy6*67, p<*001 

The lecture was preferred to the program in giving a better 

sense of direction* ^(51) = 2*51, p^*05. 

Religion Program * Refer to Table 5 in Appendix E-7 for a 
summary of mean ranks for all students participating in one of the 
istudies involving the religion program* Discussion was found to be 
more difficult to review than other methods of instruction* 

Analysis of student responses on the questionnaire revealed 
:hat students believed the program provided a better sense of direc- 
ion than class discussion, (101) = 2*99, p<|[ *01 and was easier to 
review, t(94) s 9*11, p<*001* 

Xn the opinion of the students, the lectures were paced better 
:han the program they used, t(83) * 3.07, p<*01, while the program 
uas easier to review than lectures t(83) & 2.15, p<*05* 
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political Science Program . Refer to Table 6 in Appendix E-7 * ' 
for a summary of student attitudes in regard to the material cover- 
ed in the political science program. Students felt that discussion 
was more difficult to review than other methods of instruction. 



Analysis of the data revealed a significant difference in 
nean rank of student responses to programed inaterials and discussioi 
as a method of instruction. The students preferred the program to 
discussion as a positive learning activity t(53) = 2.03, p< .05 and 
believed the program was easier to review ^tfso) = 4.36, p< ..001. 



No significant differences in mean ranks occurred when com— 
p>aring this particular program with lectures covering the same 
naterial. 



description and Analysis of Results of the Individual Studies 



STUDY 1 






Evaluators* Ray Mizer and Robert Johnson, English Dept., De 
auw University. 






Course in which program was tested: Two sections of Intro- 

luctory Poetry. 



Progrhm: Poetry: Method and Meaning by James Cook. 



Dates of Experiment: September 13 - 27, 1965. 



Design: Number 1. Group A was given the pri^gramed material, 
'3roup C received expository lectures. There was liq Group B. 



Details of Evaluation Design: Professor Mizer was in charge 

of the programed material group and Professor Johnson delivered the 
expository lectures to Group C. These were two separate classes. 

f he students worked on the program completely on their own time out- 
ide of class. The class discussions were taped and the lectures 
were taped. ^ The evaluators constructed a content test, 50 minutes, 
half objective and half essay over the material covered in class 
discussion during the second week. Each student was asked to write 
ive questions about their reaction to the material covered in the 
rogram and the lectures. 



Results: The method o'f presentation was confounded with the 

rofessor variable. Further, a preliminary analysis of pretest 
cores indicates that statistically, the two groups were drawn from 
ifferent populations. Therefore, valid comparison of methods by 

naly.is of variance* cannot be nade In this particular case. 



y" * ' ^ ' ' ' ■ — - 

For statistical procedures used in the analysis of variance 
See Appendix E-10. 
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However 9 the differences between the pre and pOb>c test scores indi- 
cate that both groups learned from the method of presentation used» 
either programed materials or lectures. See Tables 1 a»b and c in 
Appendix E8 for summary of data. 

The attitudes of the students were analyzed and the only 
significant differences in mean rank was in regard to review, where 
the program was judged to be easier than class discussion, t*(23) » 
2.10, p< .05. 

STUDY 2 

Evaluator: Philip Church, English Dept., Kenyon College. 

Program: Poetry: Method and Meaning by James Cook. 

Course in which program was tested: Two sections of Freshman 

English. 

Dates of Experiment: November 16 - December 16, 1965. 

Design: Number 1. Group A received the program while Group 

; rtotivid liotuptg ovtp tht ginii Thtrt Nig no Group Bi 

Details of Evaluation Design: The two classes participating 

Ln the study were divided in half *with one-half receiving the pro- 
.;ram and the other half the lectures. The discussions in class were 
taped as well as the lectures. ‘ t' ■ 

Results: The amount of learning for both t||!| program and lec- 

;ure groups was significant, p<*001. Students l^grned as well by 
the method of program as by lecture method. No interactionlB f^as 
i»vident. Summaries of group means and the analysis of variance are 
];>resented in Tables 2a and b in Appendix E-8. 

The analysis of student attitudes in this study indicated no 
iiignificant differences in mean rank in groups comparing programed 
instruction with other methods of instruction. 

15T0DY 3 



Evaluators: William Judd and Charles Weis, 'English Dep^t. , Ohio 
Uesleyan Oniversity. 

Program: Poetry: Method and Meaning by James Cook. 

■' . .n i l -I — « ■■■ 

V t biased on correlated observations 

No interaction indicates that the groups did not learn at 
significantly different rates. 












mim^ 












0 ^ 



Couix*se in which program was tested* 
Poetry. 



Elective Cotiirse in 



Dates of Experiment: September 22 - October 13, 1965. 

Design: Number 1. One group was given the program, the othex 
group received lectures over the same material* 

Details of Evaluation Design: The instructor in each sectlov 

asked for volunteers to work through a short supplementary unit on 
poetry outside of class. The groups were very small with few vol- 
unteers so the volunteers and non-volunteers were combined in each 
class. Professor Weis was in charge of the program group and Pro- 
fessor Judd in charge of delivering the lectures to the other group 
The lectures and class discussions were taped. The two evaluators 
collaborated in constructing a 40 minute content test, half esSay 
and half objective which was given to both classes, in addition to 
the pre and post tests used in the evaluation. 

Results: The amount of learning during the study for both 

the program and lecture groups was significant f p< tOOl, The stu- 
dents learned as well by program as by leeture* No interaotion 
was evident* See Table 3a and b Appendix E-0 for data summaries* 

The analysis of student attitudes in this study revealed no 
, significant diffi|rences in mean ■ rank between programed instruction 
and other methods' of instruction* 

STUDY 4 

Evaluator: T. R. Burkett, English Dept., Denison University. 

Program: Poetry: Method and Meaning by James Cook. 

Course in which program was tested: Two sections of Fresh- 

man English* 

Inclusive dates of experiment: January 3 - 19, 1966. 

j 

Design: Number 2. Group A» program followed by discussion. 

Group ^B, the program followed by lecture and instructor-directed 
question-and-answer period. 

* 

Details of Evaluation Design: The discussions were taped, 

the evaluator constructed a 50 minute test over the content covered 
in the three periods devoted to it. Test was half essay and half 
objective* 

Results: The amount of learning during the study for both 
















the program discussion and program-lecture groups was significant 
p<.001. No significant differences in learning were found in the 
methods of Instruction used; there was no significant Increase in 
learning after lecture or discussion. The analysis of variance re* * 
vealed no significant interaction. See Tables 4a and b Appendix El 
for details. 

In analysing student attitudes toward different methods of 
instruction it was found that students preferred the programed 
materials to discussion for review purposes* t*(32) c 4,17, p^ .OOL. 

STODY 5 

• 

Evaluators Thomas Boyle, English Dept., Albion College. 

Program: Poetry; Method and Meaning by James Cook. 

Course in which program was tested: Two sections of Amerlcar 

Literature. 

Dates of Experiment: November 22 - December 16, 1965. 

Design: Number 3. Group A, program followed by lecture; 

Group B, lecture followed by program. 

Details of Evaluation Design: Two content tests were written 

by the evaluator each 50 min., half essay and half objective. 

These were administered in addition to the pre a^d post tests giver 
to all participants. The discussions and lecturii| were taped. 

C4 * 

Results: The amount of learning was significant for both 

groups p<ii^ .001. The students learned as much whether the program 
was used before or after supplementary lectures. The analysis of 
variance revealed no significant Interaction. See Tables 5a and b 
Appendix E-8 for details. 

The analysis of student responses on the attitude question- 
naire revealed two comparisons of teaching methods which were 
statistically significant. The lectures were preferred to the pro- 
gram on the question asking students to rate each method of in- 
Istruction as a positive learning experience, t'* (52) - 2.09, 
p^.05. The students also expressed a prefere'nce for the lectures 
over the program in regard to the sense of direction that each 
method conveyed, t* (51) = 2.05, p<.05. 

STUDY 6 



Evaluator: Daniel Anderson, Philosophy Dept*, 

University* 

a — ■ ■■ '" — ' 

t based on correlated observations* 
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Program: Language of Logic by Morton Schragrln. 






Course in which program was tested: Two sections of Intro- 

duction to Philosophy. 



Dates of Experiment: first half - Sept. 20 - Oct. 8, 1965; 

second half - Jan. 4, - 10, 1966. 

Design: Number 1. Group A was given the program; Group B 

was given text material covering the same content. 






Details of Evaluation Design: One section was used for the 

experiment during the fall term. It was split in half with the 
first half receiving the program and the second, the text material. 
The same was done with the one section using the experiment during 
the winter term. The two program groups and the two textbooks, 
groups were then combined; students were assigned to the groups 
on a random basis. The textbook version was written specifically 
for this project following the content of the program that was 
being evaluated. The class discussions and lectures were taped. 

Results: The amount of learning was significant for both the 

group using programed materials and the group using regular text 
materials, .001. The students using the program learned as 
much as the students using the text materials; apd the analysis of 
variance revealed no interaction. See tables 6a and b Appendix E-8 
for details. ^ 

The analys|.s of student attitudes revealed that students pre- 
ferred programed materials to discussion for review purposes t* (11) 
= 3.05, p<.01. r ^ 



STUDY 7 



Evaluator: Brad Angell, Philosophy Dept. Ohio Wesleyan 
University. 

Program: Language of Logic by Morton Schagrin. 

Course in which program was tested: Two s'ections of Intro- 

duction to Philosophy, one in the fall quarter and one during the 
winter quarter. 




o 
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Dates of Experiment: First half, October 16-28, 1965; the S€»- 
eond half, January 21-February 11, 1966. 

Design: Number 1. Group A, program material; Group B, text- 
book material. 

g " ""' ' "" ■ ■ M , 

t based on correlatdd observations. 
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Details of Evaluation Design: Half the students in the fall 

section were given the programed unit and half the text material* 
The same was done with the other section* The groups were split 
randomly. The textbook material used in the study was taken from 
Professor Angell's own te^xt on Logic* The class discussions for 
both groups. were taped. 
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Results: The amount of learning was significant for both the 

program and the textbooks groups, p<. 001 * No significant 
differences in amount of learning favored one method over the other* 
Students learned equally well from both program and text materials* 
Also the analysis of variance revealed no significant interaction* 
See Tables 7 a and b Appendix E-8 for summaries of the data* 

The analysis of student attitudes revealed a preference for 
the programed material over class question and answer discussion in 
the areas of positive learning experience t*(9) = 4 . 98 , p<*001; 
sense of direction t* (g) s 2 . 64 , p< * 05 ; and ease of review, t* 

(6) = 4 . 25 , p<*. 001 . The students expressed opinions favoring pro- 
gramed instruction over the textbook as a positive learning ex- 
perience, t**( 42 ) s 2.55, p<. 01 . 



STUDY 8 



Evaluator: Philip Van Eyl, Psychology Dept* , Hope College* 

Program: Language of Logic by Morton Schagrin* 

Course in which program was tested: Introduction to 

Psychology. 

Dates of Experiment: Sep temb er 27 - October 8, 1965 . 

Design: Number 1* Group A, programed matf^pials and Group B, 
text materials* 

Details of Evaluation Design: The text material used was 

specifically written for this experiment and baaed upon the content 
of the program being evaluated* Each of the two sections were split 
randomly into groups A and B. The students worked Independently on 
the program outside of class. The class discussions were taped. 

Results: The amount. of learning was significant for both the 



t based on correlated observations 
** t based on independent samples* 
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program and textbook groups* p<.001. The students learned equally 
well by program or textbook since no significant differences in 
method were found* but the analysis of variance Indicated a signif 
leant interaction* p^.OS. The interaction reveals that the rate 
of learning under the method of programed Instruction was faster 
than the textbook method. See Tables 8a and b Appendix E-8 for 
details. 



' 



The analysis of student attitudes revealed the program was 
preferred to classroom discussion as a positive learning exper- 
ience Jt (24) * 3.46* p<.001 and for review purposes ^*(11)* = 3.64, 
p^^.OOl. The program was also preferred to the textbook material 
as a positive learning experience* t**(60) « 3.00* p<.001. 



STUDY 9 

Evaluator: Robert Do Haan* Education Dept.* Hope College. 

Program: Language of Logic by Morton Schagrin. 

Co'iarse in which program was tested: Educational Psychology* 

Dates of experiment: October 5 - 19* 1965. 



Design: Number 2. Group A* program then review lecture; 
Group B* program then discussion. 



Details of Evaluation Design: Students worked on the program 

on their own time outside of class. Professor Schagrin* writer of 
the program* conducted the discussion session aqd‘ also delivered 
the review lecture. The lecture and discussion were taped. 






Results: The amount of learning for both the program + dis- 

cussion group and the program + lecture group wqp significant* 
p<.001. Mo significant differences in learning! due to method were 
found. The data reveal that the lecture group learned more from 
the program than did the discussion group but the latter caught up 
as a result of what was learned during the discussion. The net 
result was that both groups learned the same amount of material 
during the experiment. The analysis of variance revealed no signif 
leant interaction. See Tables 9a and b Appendix E-8 for details. 



The analysis of student attitudes revealed the program was 
believed to be a more positive learning experience than listening 
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t based on correlated observations 
t based on independent samples. 
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to th« loctureSf t*(27) 



•OOX and had aanae of direction in the opinion of the atudenta* t 
(28) 8 2.53» p<«01. 



Evaluator: Horton Schagrint Dept* of History of Sciencest 
)enison University* 

Pr©gram: Language of Logic by Horton Schagrin* 

Course in which program was tested: One section of Introduc- 

tory Philosophy* 

Dates of Experiment: Sept* 17 - October 4, 1965* 

Design: Number 3* Group A« program followed by supplementary 

Lectures; Group Bt lectures followed by the program* 

Details of Evaluation Design: Half of the class was given the 

program first while the other half received the lectures first » ther 
:he methods of instruction were reversed* The groups were deter- 
nined randomly* The students worked on a program outside of class* 
The supplementary lectures were taped* 

P.esults: The amount of learning for both groups was signifi- 
cant » p< *001* The amount of learning was the safne whether the pro- 
gram was used before or after the supplementary lectures* The 
, inalysis of variance revealed no interaction* 

The analysis of student attitudes revealed no significant 
differences between the method of instruction used in the study* 
dee Tables 10a and bt Appendix £-8 for summaries of results* 



Evaluator: Jerome Tovo» Philosophy Dept.t College of Wooster* 
Program: Language of Logic by Horton Schagrin* 

Course Isi which program was tested: Introduction to Philosoph 



STUDY 10 



STUDY 11 



Dates of Experiment: Sept* 25 - Oct* 2t 1965 
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based on correlated observations 
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Design: Number 4. Group A, program} Group B, lectures, 
/olunteers only were used in this experiment. 










Details of Evaluation design: Half of the volunteers had the 

programed materials and half attended lectures covering the same 
naterial. The students were asked to volunteer for a supplementary 
init on Symbolic Logic without being told which method of instruc- 
tion would be used. They were asked to participate in this experi- 
nent outside of regular class time. The lectures were taped. 



Results: The amount of learning for both the program and the 

lecture groups was significant, p<.001. No differences in learn- 
ing owing to method appeared. Students learned as well by program 
IS other methods. There was no interaction. See Table 11a and b 
ilppendix E-8 for summaries of data. 

The analysis of student attitude questionnaires revealed that 
students considered the lectures paced better than the program, 

I (4b) s 2.05, p<.05. Also the lectures wore preferred to the 
)rogram for review purposes £**(46) = 2.12, p<.05. 




ITODY 12 

Evaluators: William Stephenson and Louis Wilcox, Biology Dept 
larlham College. 

t * 

Program: Bjo-Chemistry for Biologists by W^^liam Stephenson. 
Course in which program was tested: General . |iology. 

Dates of Experiment: October 1 - November. 5, 1965. 



Design: Number 1: Group A, program} Group B, text material} 

Group C lectures. 

Details oi Evaluation Design.: The evaluators consulted 

numerous texts and consolidated a list of material for the studen^s 
1:0 use as required reading in the text material group. Students in 
the program material group were on their own outside of class. The 
discussions and lectures were taped. 

Results: Three groups were involved in this study: 1) pro- 

(;ram 2) text and 3) lecture.* The amount of learning in all three 
groups was significant, p<.001. The program group was significant 
Jy better in learning than either text or lecture, p^.Ol. The 
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analysis of variance revealed 
dlcatlng the program resulted 
either the text or lectures. 
Neither the text material nor 



in- 



significant Interactions P 4 ^. 001 » 
in learning at a faster rate than 
See Table 12a and b Appendix E-8. 
the lectures could cover all the ma- 



terial in the program. Consequently • to provjLde a better basis of 
comparisons adjusted scores were obtained based upon only thvi ma- 
terial covered by each method of instruction. The program still 
resulted in significantly greater learning than either other group 

The analysis of student attitudes revealed significant 
difference in mean ranks of student responses to different ‘methods 
of instruction. As a positive learning experience the students 
preferred the program both to the textbook method t**(58) • 4.49s 
p^.OOl and discussion t" (28) = 2.18s p 4»05. Al^o the lectures 
were preferred to the textbook materials 1^**(62) = 2.21s p^^.OS. 

In regard to the pace of the material students believed the progran 
fast in comparison with class discussions (29) = 4.10s 
They preferred the pace of the lectures t^ textbook ma- 



wcmt too 

p < . 001. 



terlal ^***(60) * 2e34s p^.OS. Howevers.for review 
program was preferred to both class discussion 
.05 and the text materials 
having 



purposes s the 
t" (24) e 2.23s p< 
i,""(59) » 2.05s In regard to 

a sense of directions the students preferred the program to 



the text material Jl'^*^(88) ■ 4.68s p^T.OOl and lecture over text 
jmaterlal Jt**(61) « 3.2?s p<.01. 



STUDY 13 



Evaluator: Prank Yows Biology Oept.s Kenyon College. 

Program: Bio-Chemistry for Biologists by William Stephenson. 

Course in which program was tested: One section of Intro- 

ductory Zoology. 

Dates of Experiment: Sept. 20 - Oct. 25s 1965. 

Design: Number 1. Group A received the programs the second 
group received a combination of lectures and textbook material as 
required reading. 

Details of Evaluation Design: The group using the program 

worked cn their own outside of class. The lectures and discussions 
were taped. 

Results: Both the program and the lectures resulted in a 

significant amount of learnings P<.001. Howevers the program was 
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^ t based on correlated observations 
**t based on independent samples. 
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superior to the lectures in this regard^ «001. The analysis of 
variance revealed significant interactir>n, P^.OOl, with the pro- 
gram group learning at a significantly faster rate than the lec- 
ture group. See Tables 13a and b Appendix E-8 for details. 



The analysis of student attitudes revealed no significant 
differences in mean rank among the methods of instruction compared. 



STUDY 14 



Evaluators: William Stephenson and Louis Wilcox • Biology 

Dept. Earlham College 



Program: Bio-Chemistry for Biologists by William Stephenson 

# 

Course in which program was tested: General Biology 

Dates of Experiment: Oct. 1 - Nov. 5, 1965 



Design: Number 2. Group program followed by a discussion} 

Group B| program followed by review lectures | Group C$ program in- 
termixed with class discussion at* the end of eaclji week of the ex- 
periment. 



Details of the Evaluation Design: Group A |)ad no chance to 

discuss the material in the class until the progriim was completed} 
however, all three groups were required to hand ||| answers to re- 
view questions at the end of each unit of the program. The lec- 
tures and class discussions were taped. 



Results: Three groups participated in this study to compare 

different methods of instruction following the use of a program. 

The three groups were 1) program + discussion, 2) program + review 
lecture, 3) program t weekly class discussion (mixsd group). All 
methods resulted in a significant amount of learning, p^.OOl. 

There was no significant difference in the amount of learning using 
different methods of instruction as measured by the three tests. 
Also the analysis of variance revealed no significant interaction. 
See Tables 14a aiid b Appendix E-8 for details. 



In alalyv^ing student responses to the attitude questionnaire 
the only significant differences were in comparing the program to 
discussion. The program was preferred to discussion as a positive 
learning experience t*(67) c 6.07, p<^.001} In the sense of direc- 
tion it conveyed t*(65) = 4.00, p/ .001 and for review purposes, 
t*(S5) 5.73, p<’.001. 



A t based on correlated observations. 
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STUDY 15 



Evaluator: Wendell Patton* Zoology Dept** Ohio Wesleyan 
University. 



Program: Bio-Chemistry for Biologists by William Stephenson* 
Course in which program was tested: Zoology. 



Dates of Experiment: Sept. 17 - Oct. 2* 1965. 

Design: Number 3. Group A* program then supplementary lec- 
tures; Group B* lectures then program. 



Details of Evaluation Design: One lab section was designated 

as group A and the other as Group B. Students worked on the pro- 
gram in lab and on their own for 1 1/2 weeks. Approximately the 
same lectures were given to both groups* These were taped and the 
discussions in class were taped. 



Results: This study indicated a significant amount of learn- 

ing took place whether the program was used before or after 
■uppltmentary Itcturee* p<[^*001* No significant differences in 
amount of learning or rate of learning was found between the two 
groups. See Table 15a and b Appendix E-8 for details. 



The analysis of student responses to the attitude question- 
naire revealed significant differences in mean ranks in three areas 
As a positive learning experience* the program was preferred to 
discussion t*(51) = 9.68* p^.OOl. The question on sense of di- 
rection reveialed a preference for the lectures over the program 
t*(49) - 2.73* p<^.01 and program over the class discussions jt(49) 
~ 3.27* p^.Ol* For review purposes the program was preferred to 
the lectures Jt*(47) = 3.48* p^.Ol. 



STUDY 16 



Evaluator: Vannie Wilson* Biology Dept. Denison University. 
Program: Bio-Chemistry for Biologists by William Stephenson. 



Course in which program was tested: Freshman Biology. 



Dates of Experiment: Oct. 18- Nov. 17* 1965* 

Design: Number 4. ‘Group A* program; Group C* lectures 



*t based on correlated observations* 
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Details of Evaluation Design: Students in both sections were 

asked to volunteer for a supplementary unit of work in bio-chemis- 
try. Practically all the students in the two sections volunteered; 
consequently • there was no subdivision among volunteers and non- 
volunteers. ThuSf this study is included under Design lumber 4. 

The students worked independently on the program outside of class. 
The lectures and class discussions were taped. 

Results: Both the lecture method and the program method re- 

sulted in a significant amount of learning, p^.OOl. And there 
was as much learning by program as by lecture. However, tha analy* 
sis of variance showed the program group learned at a faster rate 
i.e. there was significant interaction, P<[.001. See Tables 16a 
and b Appendix E-8 for details. 

The analysis of student responses to the attitude question- 
naire revealed only two areas of significant differences in mean 
ranks. The lecture method was preferred to the program method as 
a positive learning experience t**(87) s 2.99, p<.01« However, 
the pace of the program was preferred to the pace on the class 
room discussions t*(25) ** 2.34, p^.05. 



STUDY 17 

Evaluator: Melvin Vulgamore, Religion Dept., Ohio Wesleyan 
University. 

Program: Studies in the Gospels by Robert Montgomery. 

Course in whict program was tested: One section of Bible. 

Dates of Experiment: Nov. 1 - 15, 1965. 

Design: Number 1. Group A, program; Group C, lectures. 

Details of Evaluation Design: Both the class discussions and 

the lectures were taped. The class was divided randomly into the 
two groups, the group using the program worked on their own outs,ide 
of class. I 

Results: The amount of learning was significant for both the! 

programed and lecture groups, p^.OOl. The program method resulted! 
in significantly more learning than the lecture method, p<.05. 

The analysis of variance revealed that the interaction was signifi-l 
cant I that is, the rate of learning in the program group was signif| 
icantly faster than in the lecture group, p^.OS. See Table 17a 
and b Appendix E-8 for details. 
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The analysis student responses to the attitude question** 
naire revealed that the students preferred the lectures to the pro*’ 
.gram as A positive learning experience ^**(37) = 3,99» p^.OOl. 

In the area of pacing of the material presentedt the students pre- 
ferred the program to the lectures indicating that the lectures 
were too fast s 3.48, P<[.01. For review purposes the 

students preferred the program to class discussion j^(14) ^ 7.62, 

p ^ • 001 • 



STUDY 18 

Evaluator: Richard Stegner, Religion Dept., Illinois Wesley- 
an University. 

Program: Studies in the Gospels by Robert Montgomery. 
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Course in which program was tested: Two sections of Religiox 

I, Introduction to Christianity. 

Dates of Experiment: Oct. 4 - 20, 1965. 

' 

Design: Humber 1. Group A, program; Group C, lectures. ' " - 

Details of Evaluation Design: Half of eao||i section was 

designated as the program group and half received the lectures. 

The selection was done randomly. The students w|irked on the pro- 
gram outside of class on their own time. The cldss discussions 
and lectures were taped. 



Results: The amount of learning was significant for both the 

program and lecture groups in this study, p^.OOl. Students learn- 
ed as well by program as by lecture method. The analysis of vari- 
ance revealed* no significant interaction. See Tables 18a and b 
Appendix E-6 for details. 



The analysis of student responses on the attitude question- 
naire revealed the following significant differences in mean ranks. 
As a positive learning experience the program was preferred to 
class discussion t/(6) “ 3.04, p^.05. No other 
differences in student attitudes appeared. 



significant 






STUDY 19 



Evaluator: Lee Scott, Philosophy and Religion Dept.^ Deni- 

son Uttivei»sity-. 

i W : ‘ ‘ ^ 
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At based on corr elat ed observat ions 
A*t abased on independent samples. 
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Program: Studies in the Gospels by Robert Montgomery* 

Course in which program was tested: Introduction to Philoso- 

phy and Religion^ two sections. 

Dates of Experiment: Jan. 3 - 21» 1966. 

Design: Number 2. Group program then class discussion; 

Group D, program then lecture and instructor-directed question and 
answer period. 

• 

Details of Evaluation Design: One section was designated as J 

group A and received the programed instruction, the other section 
received the lectures. Students worked independently on the pro- 
gram outside of class. Discussions, question and answer sessions, 
and lectures were all taped. 

Results: A preliminary analysis of sample means revealed 

that the samples were not drawn from the same statistical popula- 
tion: consequently the significant differences between the two 
groups as reported in Table 19b Appendix E-8 cannot necessarily be 
attributed to the differtnoe in methods used* Thf interaction be- 
tween groups- and test scores is also significant; 'this results fron^^ 
the fact that even though the two groups were different at the be- ij 
ginning, both of the groups demonstrated equal cq^petence on the | 
post test and the final test. The lecture group 'gained at a faster! 
rate but this made no difference on the final teaf«r' However, learm- 
ing was significant for both groups, p<.001. Se^: Tables 19 a and 
b Appendix E-8 for details. 

An anal3'©is of the studen'^^s responses to the attitude ques- 
tionnaire revealed that the program was preferred to class dis- 
cussion in three areas. First as a positive learning experience, 
the mean rank was higher, t*(i9) 5.45, p<.001; also the students 

felt the discussion was tooT slow in comparison with the program as 
far as pace was concerned t^*(i9) - 2.63, p<.05. The program gave 
a better sense of direction than the discussion i.*(19) ~ 4.51, p< 
.01. For review purposes the students also preferred the program 
to claiss discussion (19) = 5.37, p<.001. The program also gave 
la better sense of direction than the lectures in the opinion of the 
students it*(25) = 2.14, p'’^.05. The lectures were more favorably 
received Than the class discussions in all four areas. As a posi- 
tive learning experience, the analysis revealed ^**(46) = 2.70, p< 
.05. The students felt the pace of the discussTon was too slow in 
comparison with the lectures ^**(47) = 2.50, p^.OS. The lectures 
gave the students a better sense of direction than the discussion 
^**(46) » 2.44, p <. 01. For review purposes the lecture was also 
preferred to discussion jt"*(48) s 4.77, p<»001. 
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STUDY 20 



Evaluators# Richard Stegnor and Jerry Stone* Religion Deptf* 
Illinois Wesleyan University. 

Program: Studies in the Gospels by Robert Montgomery. 

Course in which program was tested: Two sections of New 

Testament Religion. Note: This was an experimental course that 

the students took exclusively during the month of January only. 
Stegnor' 8 section had had a previous course in Old Testament} 
Stone's section had had no previous course in Religion. 

Dates of Experiment: Jan. 5-7, 1966. 

Design: Number 3. Group A, program then supplementary lec- 
tures; Group B, lectures then programed instruction. 

Details of evaluation design: The students were taking only 

this one course, consequently they could work on it for eight hours 
each day. The same lectures were given to both groups and the 
students wars ixpsotsd to work on the program outside of oliss. 

The lectures and class discussions were taped. 

Results: The amount of learning was significant for both 1 

groups, p^.OOl. It made no difference whether the program was 
presented before or after the supplementary lectures. The analysis 
of variance revealed no significant differences in method, and 
there was no interaction. See Tables 20a and b Appendix E-8 for 
details. Analysis of the students' response to the attitude 
questionnaires revealed significant differences between the lec- 
tures and the program as a method of instruction. The students 
preferred the lectures to the program as a positive learning 
experience 1*(56) = 2.18, p <.05. Also the pace of the program was 
too slow in comparison to the lectures ;t*(56) = 3.00, p<.01. 
However, the program was preferred to lectures for review purposes 
i*(55) = 2.02, p<.05. Alsc the program was preferred to dis- 
cussion l.*(53) = 5.10, p<.001 for review purposes. 



STUDY 21 



Evaluator: Fred Wirt, Government Dept, Denison University. 

Program: An Introduction to Systematic Analysis in Political 
Science by Lois Pelekoudas. 



* t based on correlated observations 
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Course in which program was tested: Government 211. 

Dates of Experiment: Sept. 20 - Oct. 4, 1965. 

Design: Number 1. Group A, program; Group C, lectures. 

Details of Evaluation Design: Half of each section was 

assigned randomly to Group A or Group C. The students worked on the 
program on their own time outside of class. The discussions in cla&|8 
pind the lectures were taped. 

Results: The program and the lecture groups learned a signif- 

icant amount of material during the study, p^.OOl. The analysis o 
variance revealed no significant differences in learning resulting 
rom methods used and there was no significant interaction. See 
ables 21 a and b Appendix E-8 for details. The analysis of tha 
itudent*s responses on the attitude questionnaire revealed the 
ollowing significant differences in mean ranks. The program was 
referred to discussion for the sense of direction that it conveyed 
*(16) = p ^.05. For review purposes the program was pre- 

ferred to 

poiei tht 
: 3.69| p 



5TUDY 22 



class discussion ^"(13) = 3.77, p^.Ol. For review pur- 
leoturei were also preferred to oXano dlaouealon 
<. 001 . 



Evaluator: Sue I'^cNaghten, Government Dept., Denison University 

Program: Introduction to Systematic Analysis of Political 
Science by Lois Pelekoudas. 



Course in which program was tested: Govt. 211, three sections 

Dates of Experiment: Sept. 20 - Oct. 4, 1965. 

Design: Number 2. Group A, program then discussion; Group B, 
>rogram then lectures. 

Details of Evaluation Design: Two sections comprised group A 

nd group B. The students ^ worked on the program on their own time 
utside of class. The discussions in class and the lectures were 
aped. 

Results:. The amount of learning in this study was significant 
or both groups. The students learned as much whether the program 
tras followed by discussion or followed by a lecture. The analysis 
(if variance revealed no significant interaction. See Tables 22 a 
< nd b Appendix E-8 for details. 

t based on correlated observations 
**t based on independent samples. 
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The analysis of the student responses on the attitude ques- 
tionnaire revealed only one area of significant difference in ®«an 
rank. For review purposes the program was preferred to class dis- 
cussion t*(36) = 2.B9, p<^.01. 



Evaluator* Roy Morey f Government Dept . % Denison University. 

Program: Introduction to Systematic Analysis o f Political 
Science by Lois Pelekoudas. 

Course in which program was tested: Govt. 211, 2 sections. 

Dates of Experiment: Sept. 20 - Oct. 4, 1965. | 

Design: Number 3. Group A, program then supplementary lec- 
tures; Group B, Supplementary lectures followed by program. 

Details of Evaluation Dosign: One section was designated 

Group A, and the other section was designated Group B. The lec- j 

tures were taped, and the students worked on the program outside j 

of class on their own time., j 

Results: The amount of learning in this study was signifi- 

cant for both groups, p <.001. There was no significant differ- 
ences in the amount of learning between the two ||*oups; however, 
the analysis of variance !’'evealed a significant Interaction p\.05# 
The group using the program followed by lectures ^gained at a 
significantly faster rate than when the program |ame after the 
supplementary lectures. See Tables 23a and b appendix E-8 for 



responses on the attitude questionnaire revealed no significant 
differences in mean ranks for any of the methods of instruction 



University. 

Program: An Introduction to Systematic Analysis in Politic^, 
Science by Lois Pelekoudas. 

it t biased on correlated observations 
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details 



The analysis of student attitudes as indicated by their 



compared 



STUDY 24 



Evaluator 



William Westbrook, Economics Dept*, Denison 
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Course in which program is to be used: lPi»inciple8 of Econom< 

ic8 211* 

Dates of Experiment: Oct. 1 - Oct. 9» 1965. 

Design: Number 4. Four groups: A^BtCtD designated as follows : 



Volunteers 
Non Volunteers 



Extra Work counts 

Credit on final grade 



B 



D 



Details of Evaluation Design: 



Students were asked to volunteer for a unit on political 
science by programed materials. A similar number of non volunteer^ 
were assigned to work through the program as a supplemental unit 
to the course. Half of the participants, Groups A and C, were then 
told they would receive extra credit for the unit of work if their 
performance on the unit test would improve their final course grade 
Half of the participants were told that their grade on the unit 
test would count toward the final course grade regardless of the 
quality of their work. 

Results: Pre-and Post test scores were obtained for all four 

groups. The amount of learning for all four groups was significant | 
p/.OOl. There were no significant differences learning betweenf 
the volunteer and non volunteer groups. The pretest scores under 
required versus extra credit were significantly P< • 001 , 
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Thirt7ori thi ilgriif leant dlfferanca In laarnlng, F(1,40) • 4.7i, 
p^,0§» bataaan tha group racalving axtra cradlt and tha group 
•tth rogulrad cradlt cannot nacaaaarlly ba attributed to tha 
motivational factor of credit foi tha unit, Tha analyala of 
variance revealed no significant Interaction# See Teblee 24 a 



and b Appendix C*0 for detalla* 

The analyela of the atydenta* reaponeea on the attitude 

queatlonnalre revealed no significant dlfferencea In the mean, 
ranks for any of the motivational factors compared, 

DISCySSION 

The data substantiate clearly that studenta do learn from 



programed materials under a variety of conditions aa Judged by 
pre and post test results. Also students con Ijiorn as much from 
programed moterlala as from other methods of ln||;tuctlon. In the 
case of the biochemistry program students learn|d significantly 
more from programed materials than from either lectures or text- 
books covering the some moterlol. It should be recalled, hoeever, 
that the programs ears short ones, taking only teo to throe seeks 
of osalgnad time, eith average student time of 5 - 12 hours. 

Under design #2. «e tried to find out shot activity- 
lecture or quastlon-ond-anseer discussions— folloelng the use of 
\ : the program best capitollrod on shat eas learned from the program. 

The five*" studies using design #2 did not give any clear cut 
evidence favoring one activity over another folloelng the use of 






a program. 

The five studies using design #3 gave evidence that a pro- 
gram can be uaed effectively to Introduce a unit of material or for 
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rtvlaw purposas. Kltarlng tha saquenca af mathpda af Inatructlan 
mada na algnlfleant dlfferonca in tha aiaaunt af atudant laarning. 

Whathar atioiSlonta valuntaar ta laatn a unit af aatatlal at 
If it ia praaantad aa a saquitad part af tha cauraa mada ne 
diffaranea in tha aaaunt af laatning that taak placa baaad an tha 
thraa atudiaa that invalvad tha uaa af waluntaara. 

Tha atudant attitudaa taaard prngraaad inatroctian aata 
mara paaitiva than nagativa. In caapeting tha aathada af inattuc- 
tian awat all atudiaa, aa a paaitiva laatning txpatianca, tha ardat 
af ptafatanca fawatad lactutaa, than ptagtaaad aatatlala, than 
diacuaaiana and finally taxtbaaka. Hatavat, Tablaa B-2 ta B-6 
indicata that thla atdat vatlad aaang tha individual ptagtaaa. 

Tha ptafessata canducting tha atudiaa a|ptaaaad paaitiva 
attitudaa taaard pragranad Inattuctlan fallaaln| thait pattlclpa- 
tlan. Thay aata nat aakad ta caapata ptagtaaad |nattuctlan aith 
sthtr withodt of Instructloo* 

iULTIPLE CORRELATION ANALYSIS 

The final traataant af tha data canaiatad af caaputlng 
aultipla cattalatiana ftaa the data af each af the five ptegtaae 
that tha ptajact evaluated. Taanty vatlablaa that aata included 
in tha aultipla cattelatlan ata given belaa and aay be faund in 

i 

tholr origiool wording in ApptndlK E-6# 
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2 . 

3a 



A. 

5. 

i. 

7 0 

0* 

9« 



Sox 

Class In collogo . a.a « — * 

CEE0-W (Collogo Entranco Exarolnatlon BootOp 

Math scoros) 

CEEB-V (Vorbal) 

Pto*tost ocoro for oacb program 

Po8t«tost ocoro for oach program 

Rank in high school class 

Q»2A Amount of tlmo spont on tho program 

C-4B Evaluatlan af tha ptagtaa ftaa vaey paaitiva 

ta vary peat . 231 * . ..jg 2 
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10. Q-5A Pace of tho Pr«9**"/*“" f**i,*” ^2"f!iS* 

11. Q-6A Sanaa of direction In ueing tha program from 

••nevBT in doubt^* to •^lost most of thi tlwo 

12, Q-7A Eoii of roviow* frow vory loty to vtry 

13, Q-BA^Rat®d programs tS^oro- 

being taught from programs batter than, to pro 

K orama lass good than 

u. Q-9 Rating of couraa In ohlch program aaa uaod 
from most liked to least liked . 

15 Q-10A Effect of program on attitude toeard tho 

course from favorable to ^^^®5®^®*^i®^. 

16, Q-11A1 Evaluation of program from boring to 

17, Q-11A2 Evaluation of program from too haro to 

13, Q-llA3*Evaluatlon of program from efficient to 

19, S**1A4 Evaluation of program from delightful to 

20, Qaal2 Have you used program materials before? 

means, standard deviations of each of the varlablea and 
N»a of each o? tho five CLCA-produced programs that more evaluated 
are given in Appendix E-11. In computing the multiple R*a the 
Post-test ecore eaa used ae the criterion variable against ehlch 
the other variables eere correlated. Varlablea 3, 4, and 5 mere 
forced for each program, and thereafter the correlation proceeded 
to each of the highest partial coeffidinta until only inolgnlflcant 

ones remained. 

Results 

Th,tabl8 of partial eorralatlon coafflclanta for aarlablaa 
3, 4, and 5, and amount o^ votianca accountad for by thaaa Vor- 

lables is given ln^.Teble -fl*; . 
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Vatlabla 3« CEEB-* la aaat highly cotralatad with tha ctlta- 
rlen. tha peal taat aeotaa of tha Logie ptogtaa (2) hot la not 

.ignlfleantly tolitod to aoccoaa on tha Political Selaneo p.ogto. (S) 
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both of .hlch art reaaonaWo flndlnga. »hat la last undatatandabla 

la tha algnlflcant poaltlva corralatlon of CEEB-(ll acoraa alth 

auccaaa on tha Rallglon progran* (^)» 

Varlabla 4, CEEB-W la significantly eorralatad alth tha 

Poatry (l) and Political Sclanca programs, laaa so alth tha logic 

and Rmllglon programs, and not significantly eorralatad alth 

suceass on the Biochemistry program (2). ■ 

Varlabla 5, tha pra-tast designed for each program Is not 
significantly correlated alth post test scores on the Blochamletry 
program but Is for all the rest of the programs, especially for 
tha Religion program ahara It accounta for a good deal more of tha 

varlanct than do CEEB-WI and CECB-V# 

The Multiple R's for the three variables range from a loa 

of R ■ .1161 for tha Biochemistry program to R ■ .5203 for the 
Logic program. The amount of variance (squire of multiple R) 
accountad for by tha three variables ranges from a loe of 1.35 
percent for tha Biochemistry program to a high of 27.07 percent 

for tho Logic prograiHa 

■ Tha .tabli of partial correlation coafficiaot® aftar Var- 
labloa 3, 4, and .5 more partlallad out are given in fable 9. 
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Tabu 9-Pattlal Cort . UUor > Co.fficUnta ^ 




.07 



3, CEEB-i 

4, CCEB-V 

5, Pr®-T«st 

B . 2 



7. Raink 

HS clati 



e. Q-2A 
9. Q-4A 
10. Q-5A 



11. Q*6A 



no 



12 . . 0»7A 



,00 



13. Q-0A 

14. Q»9 

15a Q*10A 



,.19* 



‘■' i -' Vff ?''* 



• .10 



16. Q-11A1 

17. Q-11A2 
10, 0-11A3 
19. 0-11 M 



20. Post-Taat. 



Multlpla R 
*.|1* ■ Sauata. 



. 5597 



2 


3 


TT 


5 


16* 


. 06 1 


".10 


.27* ’ 


09 


.22* 


.06 


. 04 


,00 


.00 


.00 


.00 


.00 


.00 


- . 00'.’ 


.00 


i00 I 


.00 


.00 


.00 


.03 


-.03 


. , 1 6* 


..05 


.02 


-.01 


-.30* 


.13 


.01 


.06 


.04 


.21* 


,20*' 


• ,09 


• ,16* 


^ ■ 

• •09 


.06 


• 15 


— . 02 


-.10 


..17* 


• 00 


-.20* 


-.03 


. . 01 


.02 


— . 1 6* 


-.04 1 


-.01 


— .06 


-.19* 


1 —.13 1 


-.07 


.05 


.09 


.03 


-.15 


‘"TOT' 


»';‘1 4 


' a 1 9*1 


.IS" 


» .00 


e17< 


1 ,10* 


.35^ 


» ,07 


-.07 


0.11 


-.09 


.01 


-.17 


* -.03 


-.20 


* .05 


1 -.22 


* -.05 


1.00 


1.00 


1,00 


1.00 


.636 


i8 .348 


6 .575 


6 .5632 
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ir 2S 3J 4Sr 
(Hath Scora# 
Verbal Score. 
Pre-T est. 

Used PI before 



lYes 2No 
0 to 100 






Time spent on PI to 
nearest hour 10 to bl 

Evil, of PI. 1 vory 
poaltlva, 5 very poor 

Pacn. 1 too fast, 2 
about rlQlhti 3 too 
sioWf 4 uneven 

Sense of direction. 

1 never in doubt « 5 
lost moat of time 

Case of review. 1 vary 
easy, 4 very diff.* 

5 did not review 

Rate PI to texts, etc. 

1 highest, 4 lowest 

Rata course. 1 most 
liked, 2 least liked 

Eff. of PI on att. to 
course. 1 favorable, 

3 unfavorable 

Eval.of PI. 1 boring, . 
5 stimulating 

Eval.of Pla 1 too hard, 
5 too easy 

Eval.of PI. 1 effi- 
cient, 5 Inefficient 

Eval.of PI. 1 delight- 
ful, 5 irritating 

Criteri 
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For program 2, 6 warlabl.a i^ara partially eorralatad. (3 4 S 

1 7 1 9 1 1 ! 

For program 3, 10 warlablaa mera eorralatad. (3 4 3 2 10 9 19 

6 17 IB; 

For program 4, 10 warlablaa aara eorralatad. (3 4 5 7 11 6- 12 

13 12) 

For program 5, 10 variablaa aai»-partlally eorralatad. (345 
1 8 15 13 9 17 2) 

ntgcyaalon 

Tha flrat eolomn glwaa tha partial eorralatlona of tha 
17 ramalntng wKlablat alth tha erltarlon warltbla, tha Poat-taat, 
after Warlablaa 3, 4, and 5 mate pattlallad out for tha Poatry pro- 
gram. Tha eoafflelant of Warlabla 2 (r - -.25) Indleataa that tha 
loaar tha elaaa laval of tha atudant tha moia probability of hla 
raealwlng a hlghar poat-tast aeota. In othar aorda fraahman and 
aophomoraa did battar on tha podt-taat than dM Junlora and aanlora. 
Tha eoaffldlant of Warlabla 8 (r . -.16) Juat rgaehaa algnlfleanea 
and indleataa that tha amount of time apant on tha program aaa 
nagatlwaly eorralatad alth aueeaaa on tha poat-taat. Tha tandane, 
aaa for atudanta aho apant laaa tlma on tha program to obtain - 

hlghar aeoras on tha poat-taat. Tha eoafflelant of Warlabla 13 
(r » -.19) Indleataa that, those atudanta aho ratad programad 
inatruetlonal matarl.l. agalnat taxta. l.etura. and dl.euaalon 
tandad to rata tha programa mora fawotably. Tha poaltlva eertola 
tlon on Warlabla 17 (r - .16) Indleataa that tha program tandad to 

H « Tha aaednd eolomn eontalna tha parltal eorralatlon 
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coemclonts for the Logic ptogtam. Tha partial coomdant of 
correlation for th. flret varlabl. la -.16. Thla Indloat.s that 
aocceaa on tha poat-teat eaa negatively correlated elth sex, as It 
eaa codec. Male atudents tended to do better than female atodenta. 

On variable 9 (r - -.20) was found Indicating that the studonta 
tended to rate the program as a positive learning experience. On 
variable 11, the coefficient of -.17 showed that not balng in doubt 
about the direction of the program varied with success on the post- 
teat. Variable 16 (r . .16) ahows a positive relationship between 
rating the program atlmulatlng and success on post-test scorei and 
VarltblM 17 (t a .36) IndtcitM thtt MMluitlno tht pmbum •• 
being easy tended to be associated with success on the post-test. 

In addition, Variable 19 (r . -.28) shows a relationship betwean 
rating the program on the "delightful" side of the scale and 
SUCC888 on tlh8 p 08 t-t 88 ta 

The Biochemistry program had only one significant partla* 
corralatlon. that of Variable TTr - .22) with the criterion var- 
iable. the post-test, in contrast with the Poetry program, the 
Blochemletry program favored upper classmen. The positive correla- 
tion with the criterion Indicated that upper level student, tendeo 

to receive higher scores on the post-test. 

The Religion program. «o. 4, had the largest number of 

algnlflcant partial correlations of variables with the criterion 
variable. Varlablb 6 (r - .16) indicated a relationship between 
not having used programed Instrgctlonal materials before and 
aueceee on the poet-teat. Conceivably thla ...uU might ba related 
to tha fact that the Religion program had a format that waa most 
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radically dlTfarani fron typical programad format* Variable 7 
(r • -.30) ahowod a ralatiooahip bataaan tha rank in gradyating 
class in high school and poat-tast scoras, thooa tending to be 
toaard tha loaar side of tha rank doing batter on the post-tost, 
indicating the possibility that tha program favored tha someahat 
aioaar atodant* Variable 11 (r « -.28) indicated a relationahip 

bataaan a stodent's feeling that he aas never in doobt about hia 

1 1 ,, ,*• 

sense of direction ahan using tha program and his succoss on the 
post-teat, £asa of ravida, Variable 12, (r ■ -.16) and rating 
programed materials more favorably than other mathoda, Variabla 13 
(r • -*19) aara also corralatad aith suecaaa on tha poat-taat. !n 

t * 

Variable 16, (r a *17), rating tha program as being stimulating 

> Ji * « I • " 

aas correlated aith success on tha post-test, as aas Variable 18 
(r tt -«17) evaluating the program as efficient and Variabla 19 
(r « -.22) evaluating tha program as delightful,^ 

Program 5, Political Science program, had six significant 



partiaL corraiationse Tha first variable, the sax of the student, 

fi ^ 'i *- .. ' ■■ 

shoaad a positive correlation betaean being a female student ancf~ 
success on' the post-test (r « *27) in contrast to tha Logic pro*^ 
gram ahera man students did batter. It may be significant that the 
author of tha Politicel Science program is a aoman* Variabla 8 

f V ‘ 4 

(r » *21) tended to shoe a positive correlation bataaan the amount 
of time spent oh tha program and success on tha post-teat* Thia 
is in contrast .. to ‘ the Poetry program ahich shoaad a negative 

'■ “ ■" f V'-' ' ' ■ ' ■ ■ ■ " 

corralatiowrbitaaan' time, spent on the program- and success, on thi[,.,.^..; 
poat«tait/»ttaai4bia is (r • -.19) la to bo intarpratad tn.maans 
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that favorable attitude toward the course as a result of using 
the program tended to be related to auccass on the criterion var< 
iabla* Variable 16 (r « .16) showed some relationship between 
evaluating the program as stimulating-^ and high score on the post* 
test* 

Summary of amount of variance accounted for by Variables 
3» 4« and 5, and the remaining variablesi ij given in Table 10* 

Table 10**»Varianca Accounted for by Variables 



1 


1 . , .Prooram., : . 




1 1 


^ 2 


3 


4 


5 . 


1>0 J. jl 

Percent of Variance 
Accounted for by. 
Variables 345 > 


IS. 98 


27^07 


1,35-. 


13.64 


18.89 i 

4 


Percent of Variance. . 
Accounted for«v^by ' " 

Remaining Significant 
Variables 


I * 

IS. 35 


13.48 


IQ, BO 


19.83 


12.83 


Total percent of^Ver- 
iance AccountedvTor 
by Multiple R'*’i - ■ * | 


31.33 


4C.55 


12.15 


33.47 


31.72 
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The total.'' variance accounted for by the Multiple R for. each 
program is 31.33 jsercent for the Poetry program, 40.55 percent for 
the Logic program, 12.15 percent for the Biochemistry program, . 
33.47 percent for the Religion program, and 31.72 percent for the 

a 

Political Science program; 

In the. Logic and the Political Science programs the great*> 

- *.* 

eat percentage of variance is accounted for by Variables 3, 4, end 
5. The opposite is true of the Religion and Diochemietf|' programs^ 
where the greatest percentage of variance is 'accounted for by the 
reneinlng eignificent veriebles. Varienco on the Poetry program 
^JLe;%lggoyl|tiii'1fdr'^•ebout"'iguelly h>^;:.hdth:'iete;,Pf vertebime4:j^si%^ 
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Tho Poatry program appearod to ba tha aaaieat program» 
appropriata for yndarclasamen, yot favorably comparad to otSiar 
methods of inatruction. The Logic program aaems to be wall suited 
for men studaots, and tended to be rated as easyt deli^tful, and 
a positive learning experience by those aho also scored high on 
the criterion* Mathematical aptitude accounts largely for success 
on the Logic program. The table of partial correlation coefficients 
yields little information about the Biochemistry program other than 
that the upperclassmen tended to do better on the post-test than 

did the underclassman, indicating perhaps that it is a demanding 

* ^ 

program* The Religion program tended to be easy for slower students 
and was generally rated favorably* Both Religion and Political 
Science programs probably built more on -students' previous know- 
ledge than did the other programs* The Political Science program 
favored women students and those who spent more time on it thar. 
those who did not* .It was the only program to affect the student's 
attitude^ positively. toward the course, and along with. the Religion 



and loglO/'progtawa, . was considered stimulating* 
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CONCLUSIONS AND IMPLICATIONS 

< 

That atydenta iaarnad from programad Inatructioo aod 
oxpraaaod favorabla attitudaa toward It, and that profaaaora 

L. 

waro favorably Inpreaaad with tha programa uaad, indicata that 
programed Inatryction la worthy of continued conaidaratlon po a 
lagitlmata medium for teaching at tha collage level under condi* 
tlona comparable to thoaa of tha evaluation daalgna# Programa 
were auccaaaful for differing raaaona* Given appropriate informa* 
tion about atudantai it ia poaaiblo to aacartain variablaa that 
are aaaoeiatad with aueeaaa in uaing tha^ program ae maaaurad by 
■ the erita'rlon viriabla, the poat^taet* ^ . 5 ;■ 
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THE PROCESS OF PREPARIMG PROGRAMED MATERIALS 



AS A MEANS OF ILLUMIMATIMQ TEACHING 



Ths arrival of programed instruction on the educational scene 
with the accompanying process of preparing programed materials ^sas 
made possible the radical development of an important aspect of the 
total process of teaching - - the planning of learning activities. 
This is not to suggest that planning for learning was not an impor- 
tant part of the instructional proce.ss prior to the advant of pro- 
graming. Preparing programed material as was done on the GLCA* Pro- 
gramed Instruction Project, however, involved a quality and inten- 
sity of planning that goes far beyond anything done in the past in 
the traditional approach to college teaching. Robert Gagn©^^ calls 
this kind of planning predesigning of learning conditions. Pro- 
graming has thrown into bold relief the difference between two 
methods of planning or designing learning conditions: the tradi- 

tional extemporaneously designed learning conditions, and predesigne< 
learning conditions. 



18 



As Gagne states, learning conditions can be predesigned (and 
thus, in a sense, become a part of planning), or they can be de- 
signed to meet each instructional situation as it arises. 



One of the most fundamental methodological questions that a 
teacher now needs to answer is the extent to wh'ch the learning 
situation cart be and should be predesigned for the individual learn- 
er and the extent to which it can be created extemporaneously , 
while instruction proceeds over a period of time. 



.19 



Gagne points out that extemporaneously designing the condi- 
tions of learning while the teacher is interacting with a student or 

students is undoubtedly c o n s id er ed Fy ma ny teachers to be one of 

their most important functions. ' Man./ instructors consider personal 
involvement with students to be the heart of the matter of teaching. 
It is on their skill in extemporaneously designing the conditions of 
learning that they pride themselves. It is in such activities that 
some of the deepest satisfactions from teaching arise. Extemporan** 
eously designed learning conditions constitute what might be called 
the art of teaching. 
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Extemporanciously designed learning conditions have certain dia- 
advantages • however* It is peculiarly difficult to assess the effec;:- 
iveness of extesnporaneously planned conditions of learning or to 
clarify what kinds of learnings are enhanced by such teaching* It i(i 
also difficult to train a person in techniques of this art* 



Although predesigned learning situations as exemplified in 
programed instruction are in many respects the polar opposite of ex* 
temporaneously designed situations* the two are not mutually ex* 
elusiij^e* In fact* they complement each other* Just as there are 
advantages to the extemporaneously designed conditions of learning* 
so there are advantages to t >e predesigning of learning conditions* 
Gagne^^ states the following: 



The selection of proper learning conditions can and may 
be made as an unhurr5f>d choice rather than in spur of the 
moment decisions* 



A quality control of the learning conditions is insured 
and maintained* Quality does not suffer from variations 
in teacher's skills* 



Predesigning makes possible pretesting j whether or not a 
set of learning conditions has been correctly chosen and 
designed can be determined by trying it out on the stu- 
dents and revising if necessary* 



4, 



Predesign of learning conditions greatly reduces the 
necessity for the teacher to use valuable time in extem- 
poraneous design and thus makes it possible for a proper 
emphasis to be restored to the teacher functions of 
managing instruction * motivating t generalizing * and 
assessing* 



The iilCu!nittation of hitherto relatively unexamined aspects of 
teaching was one of the unanticipated by-products of the Project* 
Little research has been done on the relationship of prenaring pro- 
gramed material to broader aspects of teaching. Lysught^^ showed 
that teachers trained to prepare programed materials used more pro- 
grams subsequently in thair classroom work express ed more favorabla 



o 

ERIC 



'i; 

iii. 



iii 






attitudes toward programs* found programing intriiijg)lcally satisfying* 
go .control group was used in the study* Testimonial " evidence a- 
bon'nds as to tha salutary effects that preparing programed materials 
has on tha understanding of teaching* 
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Jerome Lyeaught* "Inducing Classroom Change Through Pro* 
gramer Training*" Programed Instruction Bullatin Vol V, 
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Th6 remaining chapters will report more systematic evidence 
that principlea and techniques of programing have broad applicability 
to other aspects of teaching. 

The following assertions seem warranted and will serve to 
iiimmarise the material that will be presented in this chapter and 
:ested more systematically in the following chapters. 

1. The process of preparing programed material can be describ* 
ed in at least a schematic manner. 

2. Preparing programed instructional materials-a method <of 
predesigning learning conditions-sheds light upon other 
major aspects of teaching. 

3. The principles and methods used to prepare programed 
materials on the Programed Instruction Project differ in 
many respects from earlier principles and procedures. 

The Process of Preparing Programed Material 

I * 

i - ■ , 

i The 6LCA Programed Instruction Project asked more of program 

writing than the mere production of programs effective as they might 
i»e as teaching devices. The Project sought in the p:i|eparation of 
^rogrampd instructional material, a process that woitid generate new 
Insights into some ^f the fundamental dimensions of '^iteaching. If the 

I1 roces8 of writing 'j^rograms could be made to illumi^l the broader 
omain of teaching -|:he value of the process would dg^end beyond the ^ 
ere preparation of programs Into the very teaching J|rocedures used 
y the programers As college instructors. In addition » programed in* 
truction itself would be validated at least in pari',|b>' its contri- 
ution to the total teaching effectiveness of the authors. 

An attempt was made to find consultants for the training work* 
hops provided for the programers during the summers of 1964 and 
965 who would be able to provide the training experiences that would 
Ao beyond the simple production of programed material that would 
generalize to the' broader aspects of teaching. Such consultant help 
was made available to the project by the Center for Research on 
beaming .and Teaching of the Sjniversit,., of Michigan, which set up the 
-raining workshop in, the Summ^^r of 19®4 and provided editorial ser* 
^'’ice to the GLCA Programers. In 1965 the training workshops were 
conducted I by the staff members from within the Great Lakes Colleges 
iLssociation. 



Th« following' is a generalized scheme to describe how pro* 
framed instructional materials were prepared on the. GLCA Programed 
Instruetibu P^pject* . Like all^ sphimatic descriptions , however, it 
does not . ly •^I^i^ThSt^riad^^ ^>ahould -Liar in. mind-’ that 
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there were many exceptions to the procedures as outlined below* We 
soon discovered that there is no single universal approach to the 
preparation of programed materials and that there were as many in- 
dividual variations from the general scheme as there were programers 
Although the programing done on the 6LCA is not unique in toto, it 
has some features about it that are. 



Ordinarily the first step in preparing programed material is 
taken when the programer delineates in a rather^_general way the 
topic or subject he wishes to program* The first step may be no 
more than giving the forthcoming program a tentative title and block 
ing out some of the areas of knowledge to be covered by the program 
It was not unusual to find at this stage that the programer under- 
estimated the complexity of preparing programed material and aspired 
to program much more material than was feasible in one summer's 
time. When this happened, the programer was advised to begin to 
program only a small part of the content he planned to cover. 

The programer took the second step in programing when he for- 
imulated the objectives of his program. The purpose of stating ob- 
jectives is to set up a guide for the selection of material to be 
included in the program and to establish a standard of performance 
which the student is expected to be able to meet at the conclusion 
of the program. If this purpose of starting objectives is to be ful- 
filled, objectives need to be stated in much morefdetail than is 
generally done in teaching. Much more attention 'needs to be paid to 
formulating the objectives In terms of student behhvior and perfor-. 

ance than - is usually the ease.— -Frograraers in the-GLCA project 
|found that their broad overall objectives needed be broken down 
.into a more detailed outline of objectives, state|*. in terms of what 
students were expected to accomplish, and- formula4ed insuch a way 
that the program content could be derived from them. 
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Hager's book PREPARING OBJECTIVES FOR PROGRAMED INSTRUCTION^^ 
was useful in helping many faculty members catch the idea of writing 
objectives precisely and in verifiable terms* 

Experience on the pro'ject has shown that college professors 
typically stated their objectives in terms that were much too gross 
to be useful in directing what and how they taught. They often in- 
cluded objectives that were far broader thah'*t;hey could possibly ex- 
act. students to attain. Although there is no objection against 
roadly formulated objectives as such, they need to be made much mor<i 
pacific than is generally the case if they are to serve as explicit 
uides to the instructor's content and teaching procedures. 
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Robert Hager, Preparing Objectives for Programed Instruc 
tion « . San rranclsco: Fear on, 1962* ’ ' 
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FurthermoiM», tfie college teachet? often mistakes teaching for 
learning* He often states as an objective what he will do rather [ 

than what students will accomplish* 'Rather than stating what he , 

wants the student to know and be able to do at the termination of 
the course he specifies what he as an instructor will do during the 
course to help the student arrive at that point i 

This is not to say, however, that every program writer success*' ; 
fully formulated precise and specific objectives for his program the 
first time he turned his hand.^to it* Some of ‘them never did succeed 
in satisfactorily formulating **their obiectives in such terras** In 
one or two cases it might even have been better for the programer 
not to have been so concerned about formulating his objectives, 
since trying to do so seemed to set up a mental block that impeded 
progress on the further steps of program writing. The process of 
formulating objectives was not meant to be a fetish although to some • 
programers it became almost that* Neither was the stating of ob^ 
jectives -set up as a hurdle which needed to be completely, cleared 
before any further steps could be taken on writing the program* 

Nevertheless in spite of some exceptions it still remains true 
that no atep in the entire proceee of writing programed materiel 
made a greater impact upon the programer *s underetaniding of teaching 
than learning how to formulate specific verifiable Objectives* 

The attempt :to formulate objectives in some cases illumined 
the tentative title and outline for a programer an4'* suggested to 
him how to rephrase. his tentative title and re-deli^eate his subject 
matter* After the first more or less successful attempt to formulat^ 
objectives the programers usually went on to the neit step in their 
writing of the program to return later to the problem of formulating 
their objectives more clearly* Going back' to reformulate the title, 
and subject matter after having stated the objectives was only one 
of the many occasions in which a programer during the course of pre- - 
paring programed material reorganized what he had previously written 
in the light of new insights gained in taking the f*>rward step. The 
preparation of programed material consisted of a moving up and down 
the line readjusting, reorganizing, redefining, rather than a precise 
neat and logical unfolding of the material* 

The next formal step in the preparation of programed material 
was the construction of two comparable forms of a test, called a 
'"criterion test," that were suitable to assess the success of the 
student in attaining the objectives of the program* By taking this 
.step the programer came to grips with the directing function of his ~ 
objectives* That Is, if he really meant that *a student taking his - 
program was expected to attain the objectives he had formulated for 
the program i then it followed that the programer should establish 
testing procedures to ascertain whether the student had in fact 
attained the objectives* 



Th 0 instructor was introduced* perhaps for the first time* to I 
all the bugaboos of testing - validity* reliability* objective versuj 
subjective questions* Construction of adequate tests is a complex 
technical skill which most college instructors do not possess and ' 

[ cannot be expected to develop in the short time usually available to 
them for writing i>rograms. The, very process of preparing adequate 
test materials deserves a training workshop in its own right, 
iequently* the instructor's success In producing a sophisticated 
instrument for assessing student's progress toward the objectives of 
his program is likely to be limited. None-the-less * the attempt to 
develop such tests probably helped the programer develop a clearer 
perspective of his objectives which he now stated in even more 
verifiable terms. 




One form of the criterion test served as a "final exam" of the 
program. The second form of the test could servo quite a different 
purpose* although not every programer used it so. The criterion 
test was constructed parallel to the outline of objectives. The tee 
itself can become a second outline of the program called the 
"Criterion Frame Outline." Each question of the test is designed to 
ascertain the student's success in attaining a specific objective. 
The test can therefore be taken apart, question by question* each 

g uestion serving as a check point for a specific objective* to 
easure the student's progress as he proceeds through the program. 
The last item on the test serves as the last "criterion frame" of 
the program. The first test item is the first criterion frame. 

)oth the outline of objectives and the series of test items imbeddeq 
In the program serve as the framework around whlqh the program is 
uilt. As the s|udent works his way through the program both he an^ 
he instructor can gauge his progress by his success in answering 
he criterion frames whicfi serve as checkpoints the objectives 

and form the outline of the program. 

The final exam could serve not only as the test of the success 
3 f the student in attaining the objective* but cquld also bo used a| 
1 pretest to determine the student's initial knowledge of the mater*** 
Lai in the program. He could possibly be excused from taking those 
oarts of the program in which he demonstrates his initial profici* 
sncy. In practice* however, the instructor may wish a student to 

f ' ake the entire program reviewing those parts on which he demon- 
trated his proficiency. 

I Experience in training programers suggests that much more 

attention should be paid to the criterion frame outline than is 
isually the case. Brethowcr^^ points out that once the outline is 



Dale M. Brethower "Finding and Using Criterion Frames" 
“^fielected MSPI "TaDcFTTCfeSl Occasional Paper Ho. V, Ann Ar> 
University“fMlchigan, Center for Program beaming 
for Business* 196Sf p* 36* 
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constructod It can be used 1) to make a "market survey" of colleagueiji 
to ascertain from them whether they are interested in having their 
students able to answer such questions^ 2) to discover critical 
skills which should be taught » 3) to determine relevant characterise* 
tics of the student population^ 4) to eliminate irrelevant material 
from the program* 5) to check the accuracy of the content* 6) to 
separate educational problems from problems of supervision* 7) to 
generate other criterion frames* 8) to enlist cooperation of others 
in designing the program* 9) to advertise the program. 

A comment on the degree of specificity of objectives and test 
items is in order. After the programer has stated his objectives in 
specific testable terms and has developed criterion test items and 
frames* he may then wish to "loosen" his objectives by stating them 
more broadly and by making his test items on the criterion test more 
general. The purpose of loosening is to assure himself that the 
program is not just teaching the student to pass a specific test’ but 
rather to teach more generalizable behavior as well^ 

The relationship of the Outline of Objectives to the Criterion 
Test and Criterion Frame Outline is shown schematically below* Kote 
also the teaching frames that lead up to the criterion frames in the 
Criterion Frame Outline* Teaching frames will be d|acussed next* 
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Having organized his objectives Into an outline and having de- 
veloped two parallel criterion tests • one to serve as the final test 
and the other as a criterion frame outline parallel to the outline 
of objectlvest the programer was now In position to take the next 
step In the preparation of programed material, the writing of 
"teaching frames*" The teaching frame Is the unit of Information 
to wh ich the student responds* By means of the teaching frames, 
setup Inasequence, the programer presented to the student the In*- 
formatlon, problems, stimuli to be discriminated, concepts to be 
'formed or whatever the students need to know and do In order to past 
the criterion frames* The decision of what goes Into the teaching 
frame Is determined by the criterion test frame toward which the 
teaching frames are expected to lead the student* The criterion 
test In turn is determined by the objectives* Thus In a real way 
the objectives guide the selection of material to be Included In thi 



program < 



The programers at this point faced a dilemma between trying 
to motivate students and being sure that they retained what they 

»d* For the sake of maintaining the motivation of the student 



xearnaa# ror xno saKe or maxnxaxnxng xno nnuLxvauxuu va ouuuicsaii* 

the programs were kept as "lean" as possible* That Is, no more 
^teaching frames than necessary were uced to prepare the student to 
pass the test frame* For the sake of retention, however, a certain 
amount of redundancy needed to be provided In the teaching frames* 
SToo much redundancy jeopardizes motivation; too little jeopard! 



zes 



pass xne test rranno* ror uno sctKe wx ** « v c , «» v* 

amount of redundancy needed to be provided In the teaching fr 
^oo much redundancy jeopardizes motivation; too little jeopar 
botentlon* It was found that redundancy could be/provlded by 
^'branching techniques" whereby the students who desired or needed 
bore examples problems. Information, as shown by their failure to 
Ipass the criterion test frame could practice the ttesponses by tak- 
ing an extra b:;*anchlng line of teaching frames* 



Generally speaking, the program writers for the GLCA Programed 
Instruction Project adhered to the principle of obtaining active 
i?esponses from the students who were to take the program* Aside 
jfrom that principle, however, they paid little attention to set 
rules for writing teaching frames as was advocated by some program 
rlters and editors* The programers did not, for Instance, use a 
ow error rate as a criterion of the success of the program* When 
rrors did occur, it was not automatically assumed that the "step 
size" was too large* The programer sought the source of the error 
In possible ambiguous wording of the teaching frame or In possible 
inadequate sequencing of the material* In some cases earlier 
naterlal that had not been practiced enough for the student to have 
:»etalned was the cause of errors In later frames* 



Writing teaching frames was the starting point of the earlier 
generation , of programers, and Is to some extent still considered 
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the starting point for writing programs. As seen from the above i 
however, the new generation of programers as typified by the GLCA | 
programers did not write teaching frames until a great deal of pre** 
liminary analysis and tentative formulations had been completed. 



The actual writing of the teaching frames, however, served 
to illumine and clarify all the previous steps. Thus, after writ*- 
ing a sequence of teaching frames, a programer might find it 
necessary to revise his criterion frames or even his objectives. 
Although we have no experience to verify the following statement, 
it is probably true that a programer after writing a successful 
program following the ideal schematic procedure outlined her.ein 
could thereafter begin almost ut any point in the sequence with 
the writing of a program. The training program, however, followed 
rather closely the steps outlined so far in order to provide a 
frame of reference for the Inexperienced programers. 



* 

Although the content of teaching frames is determined by the 
criterion frames and ultimately by the objectives of the program, 
the sequence of the teaching frames and ultimately of the criterioi| 
frame outline itself is to be determined empirically. That is, I 
the programer as a subject matter specialist and perhaps in con- ' 
sultation with colleagues, is the person best qualified to decide 
on the nature of the objectives and the subje ct matter and sk ills 
which a student must know in order to atlbaln tffe , oFjls^tlves . 
the sequencing of the teaching frames and ultimatiply of the 



But 



criterion test items and the objectives is a pro|)lem of a differ- 
ent order whose polution is not dire tly derivable from knowledge 
of the subject matter itself. Being an expert ifi': the content of a 
discipline does not Insure the instructor's bein| an expert in 
communicating it in a learnable sequence. To th| problem of se- 
quencing we now turn our attention. 



A college instructor rarely, develops the sequence of ma- 
terial in his course or in a lecture on an empirical basis. In 
interviewing a number of college instructors the author discovered 
that it was a practically unheard of practice for an instructor, to 
try out a sequence of lecture or reading material on one or two 
students to ascertain from them whether the material was presented 
in clear and understandable form. This is not to say that instruc* 
tors do not get feedback from students. The mechanics of feedback 
in a typical college instructional setting, however, are generally 
too crude to provide an adequate means whereby an instructor can 
empirically improve the sequence of the material he presents. 
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Whero does an instructor typically find a basis for develop- 
ing a sequence of material? He can and often does find it in the 
I outline of chapters that appears in the textbook. He mav use some 
I logical basist moving from simple to complex* from known to un- 
I known* or from the presentation of principles to examples. He may 
present the material chronologically. Or a college Instructor may 
look inward and organize his ma'terlal in terms of what Interests 
him most to what interests him least* from what he is most compe- 
tent to teach to whwt he is less competent. The way the instructo 
has been taught in college and graduate school is still another 
basis to be used in organizing the material he wishes to teach. 



Two experiences of programers called into question the a- 
bove traditional approaches to sequencing subject matter. The fin t 
experience occured when the programer was confronted with the ob- 
jectives and the criterion outline he had formulated. He now was 
required to write his first teaching frame What should go into 
it? How should it be phrased? What can be assumed about the stu- 
dent? Answering those questions can and often did constitute a 
minor crisis in the programer *s life. The reason Is clear. It is 
by no means certain that we know everything that is important to 
be known about beginning a program* A programer discovered how 
many unverified assumptions he had been making about what students^ 
know* hew much uncertainty they will tolerate in getting started* | 
how much he depended on factors generated by his own personality | 
in getting a course or lecture underway. Progranling brought such j 
assumptions into the open for closer scrutiny. ^Ifuch more empiricaJ 
study needs to be made of how initially to engage a student in a 
program* or for that matter* a lecture or a courge. 



Having worked through the minor crisis of |ptting started* 
and having developed a tentative teaching frame Sequence* the pro- 
gramer encountered a major crisis when for the fllc*st time he pre- 
sented his teaching frame sequence to a real student who served as 
a test subject for a trial run of the embryo program. At this 
point the programer discovered whether or not his initial teaching 
frames and the following sequence actually did communicate to the 

i student what the programer wanted it to. Did it start the student 
on the line of thinking that would load him to the attainment of 
the objectives of the program? 



One cf the moot helpful questions a programer can ask him- 
self in beginning to write a program is ”What does the student havi 
to know in order to pass the first criterion frame of my program?” 
The steps that a student must take* the information he must possesfl 
the problems he must solve ih arriving at the point where he can 

sub.t.ne. of th« first t.aehlng 

franss. Son. oontant lands lltaelf^ to balng analysed into a seriaii 
of steps leading up to the criterion frame. The programpr can beSij 
proceed empirically in decMing how many of the steps must be in-« 
eluded* whether the firet step should be presented first to the 
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j Student I or whether he should be presented with the last step first 
land then be worked backward through the steps stopping at any point 
|at which he is able to pass the criterion test. 

However, any a priori analysis of the sequence of the material 
does not quarantee that it will necessarily mesh with the mental 
operations performed by the students taking the program. At the 
present time we know relatively little about the mental operations 
a student uses in the interactine with programed material. Bloom's 
TAXONOMY OF EDUCATION OBJECTIVES'^ , Guilford's structu re of in^ ^ • 

tellect26 and recent advances in computer simulated '^* though|) pro- I 
cesses^? give some promising leads for investigating m'eWtaX'processI* 
es involved in human learning. Programing provides the experimental 
tools for carrying out the investigation. 

There is» however, a more immediate reason for the relative 
lack of knowledge of '^ow to sequence materials in a way congruent 
to the student's mental operations. The reason is that educational 
objectives are rarely formulated for or addressed to students them- 
selves. To whom are educational objectives usually addres sed? 
Probably more to the imagined and real colleagues of >"t*he pro- 
gramers than to real students themselves. If objqctivea were ad- 
dressed to students and were meant to communicate to them what the 
program was designed to teach tUem in terms that. they could under- 
stand and visual|.ze, it might be possible for th:f' programer to go 
directly to the ftudents themselves with the objectives and ask 
them to tell him. the programer, what information or skills they 
thought they wou|d need in order to attain the onjeotives. 

This is not to say that it is a simple mattjpr to communicate 
educational objectives to students. How does a in^cher communicate 
to students what they do not yet know? In spite |f the difficul-* 
tieSf exploring the students' responses being glvCti the objectives 
for their learning should be explored and might uncover new strat- 
egies of sequencing subject matter that are more congruent with 
their mental operation than any «apr^i^ X“^ tract approaclT'tb se« 
quencing that we now use. A posii¥l¥^7idditional benefit «of ^ commun- 
loating directly to the students the objectives of the program or 
the course is that the process of communicating might tend to break 
them away from thei<* dependence (shown by so many college students) 



Benjamin Bloom, A Taxonomy of Educational Objectives 

j, p, Guilford, "Three Faces of Intellect," American 
Psychologist s 1958, 14, 469-479. 



27 Allan Ntwalli Herbert A. Simon, and John C. Shaw, "Elements 
of a Theory of Human Problem Solvings** Psychological Review s 1958, 
65. iSl-ieSs 



on tho Instructor not only for ths objectives of education but for 1 
the means of education as well* | 

Programing differs from other modes of presenting subject ’ • 
matter in that it is developed empirically* Early in the process j 
of programing typical students wars called in so that the programers 
could try out^thelr programs on them* The purpose of the programerj 
was not to defend his program against the student's criticism of it | 
but rather to invite criticism and to learn how well the program 1 
taught by observing student reaction to it* The prog'^am was pre- j 
sented to the student under conditions that resemble as nearly I 

possible the conditions under which the program will normally be I 
used as a predesigned, auto-instructional device* The programers j 
were instructed not to ad lib, suggest answers, give instructions 
tliat would be impCssible for another instructor using his program 
to give under normal conditions* The programer's main job was to 
listen, observe, take notes on how the student reacted to the pro- 
gram* When the student had worked his way through it, the programer 
I'went over it with him, rephrasing parts of it if necessary, making 
Ithe instructions still more explicit, dropping out a frame here or 
[adding one there* But the ultimate test of the program was the | 
student's success in answering the questions on the cL*iterion te^e| 
In general, if the test measures what it purports to measure M/e>, [ 
is valid) and does so under a variety of conditions (i*e*, is re- I 
liable) and if a student after taking the program passed the test 
which he could not have passed before taking the program, the pro- 
gramer is on his way to having developed an empirically validated 
teaching technique* 



The process of developing a program up to this point consti- 
tuted one of the most important phases of learni|(ig about teaching 
for the programer* Further drafting of teaching frames, reviewing 
them, trying them out both on students and colleilgues provided sti|.i 
more opportunities for the programer to learn noi^ only more about 
the content of his discipline as it is contained in his program but 
also about the learning processes which students use aj they work 
with his program* Even as specialists in their disciplines the. 
programers frequently found that they had a\ less precise understand 
ing of their material than was required by programing* 

I One of the greatly under-emphasized aspects of program writing 
fis the value of the programer's consultation with other collea gues 
|who critically evaluate the content of the program* An attempt was 
imade in the Programed Instruction Project to surround the programer 
with a team of consultants who could help him not only in the tech- 
nical aspects of- writing the program but also in the specialized 
aspects of the content of the program* Many simply procedural and 
mechenieal difficulties were encountered in trying to set up such 
a coneultentiwe teem for each programer* A more fundamental diffi- 
culty, however, grew out of the nature of programed instructional 
matettial itself • Such materials are very explicit. A programer's 
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acsumptionst his point of view, his predilections and interpreta- 
tions are immediately visible in a program. These, of course, 
constitute the sources for major differences among academic peopi^e 
As S. Narkle28 said, ”I must admit that scholars disagree -- rarely 
on facts, sometimes on definitions, often on theories, and almost 
always on interpretations." It takes a great deal of sophistica- 
tion and tough mindednesa on the part of b^th the programer and the 
consultant to develop a fruitful consultative relationship for the 
production of programs. 

When the final draft of a program was completed it was field 
tested (See Chapter 2). A field test yields different information 
from the developmental testing. As indicated in Chapter 2, field 
testing usually provides a redundancy of data, much more than a 
programer can use. By sampling the data from field testing, how- 
ever, programers found they could learn as much about the operation 
of a program as they dould in using all the data. Field testing 
also provided Information on problems of administering the program 
in a typical classroom or college teaching situation. Such infor- 
mation proved to be valuable in helping the programer write more 
adequate instructions for the use of his program. Finally, field 
testing provided the kind of information that should be presented 
to the potential consumer of the program to help him decide whether 
to use the program and if so, how tc use it. In js sense, a program 
as a validated teaching technique is comparai>le tc a standardized 
te st. To be useful a standardized test needs t(| provide informa- 
i^ion the norms of the test, that is, on the population on whom 
\tre test was standardized. Such information is lilso valuable in 
describing the purposes and the success of the py;pgram in doing the 
teaching job for which it is designed. 



In summary, what can be learned from the process of preparing 
programed material that is applicable to the more traditional 
activities of a teacher -- holding lectures, discussions, tutorial 
sessions, and evaluating student progress? Experience in the 6LCA 
Programed Instruction Project suggests that the following steps in 
preparing programed material shed light upon other teaching pro- 
cessed: 

1. Formulating objectives in observable testable terms. 

2. Writing criterion test items that validly and reliably 
assess the students' progress toward the objectives. 




Sussn Markle, "The Wastebasket Reflex. A response to Some 
Exemplsrs of the Ak>t", WSPI Journal No. 5, May 1965, p. 10« 
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3« Selecting the content of the program in ter s of the ob** 
jectivee* 





Arranging the content in a sequence in terms of our as yet 
rather inadequate understanding of the student's mental 
processesi and styles of 1^'^arning. 



5* Developmental testing of the teaching sequence ^ of the 
program - trying out sequences of teaching frames on a 
test student* 

6* Revising the contents and sequence and even critet*ion 

frames in the light of the reaction of the student *to the 
trial sequence* 

7* Drafting the remainder of the teaching frames of the pro- 
gram t trying them out on students from time to time* pre- 
senting them f^^r criticism to colleagues, revising object- 
ives and criterion tests as the response of students and 
colleagues dictate* ! 



i I 8* Assembling the entire program and ‘field testing it* I 

1^1 n 

Again it should be emphasized . that the sequence of steps above! 
i is not to be viewed as a rigid and unchanging sequence* It is sim-t 

!■ ply a 'Scheme for presenting what a programer approximately did in 1 

^ working on the Programed Instruction Project* 

I Bach of the above steps has obvious connections with what an 

5 instructor does as a lecturer, evaluator, discuggion leader, tutor, 

I It is possible for a person to teach acceptably’ for years in 

traditional fashion without ever being squarely confronted with the 
assumptions he makes about learning and teachin||;:* Writing a progr^ 
in the manner described above lead s the programi|r to examine direc^ 
ly some of the most significant aspects of his teaching processes*^: 

In the following chapter will be found statements made by GLCA 
’i', faculty members evaluating their experience as programers* These 

|i' persons prepared programed materials under the sponsorship -of the 

j>: Programed Instruction Project during the Summer of 1964. 

Contrast With Earlie r Co ncepts of Programed Instruction 

Programed instruction has been subjected to a good deal of mis- 
understanding, some of it •stemming from exaggerated claims made for 
it in. the past by over-zealous practioners, some of it growing out 
of early rather primitive concepts of the nature of programed in- 
Istruetion and from;. early exemplars of programed instructional 
hnatariaJls, some of it originating in negative attitudes of persona 
nirho opposed programed instruction and who failed to become in- 
Eormed about the changes that were occuring in the principles and 
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ipractices of preparing programed materials. 

ji The 6LCA Programed Instruction Project makes no claims for the 
(teaching value of programed instructional materials other than thos< 
jalready discussed in Chapter 7. 

Experience in producing programed material on the Programed 
Instruction Project as described in this chapter have caused the 
programers to move beyond earlier concepts of programing and there- 
by avoid some of the controversies over what now r.ppear to be irre- 
levant principles of programing. In order to highlight the .present 
position of the Programed Instruction Project with respect to^ the 
principles of programing let us look at^the most important earlier 
concepts brieflyt which* (unfortunately still prevent a few persons 
from perceiving the emerging r ^t Xo iT~Th'a F' pr o g r a m u c t i ©“h 

|is taking. 

• 

Five illustrative principles of programed instruction have bee:i|^ 
formaultsd in an effort to describe what it is and how it works 

1. The student must become actively engaged in responding to 
the program. The prinolpl^e suggests that the student 
should respond often and overtly by writing an answer* 
selecting a response from several choices* pressing a 
lever. 

2*-' The student's response should be immediately confirmed. 

The confirmation is seen as reinforcing t|ie response. 

' The student should not be allowed to respond incorrectly. 

The principle of immediate reinforcement fitated in number 
two above underscores the importance of the low error rate 
in a program since it is clearly undesirable to reinforce 
incorrect responses. 

4. The incremental steps must be small. The principle pro- 
vides the major reason for trying to reduce the error rate 
in the program. It is assumed by a programer that if a 
student makes an error on a frame of the program* the 
frame contained too much information and must be broken up 
into one or more additional frames* each of which contains 
less information. 






pw 

John Weliens* "Adjunct Auto-Instruction*" Technical Train- 
l ing and IndUstriai.-TrAihing * Evans Brothers^ blni4teiiiivl965v 'ndther 
fauthord * havelstrsssid': theses five plus IvaMationslof'^thedi and addi- 
Itional principles as well^ 
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^ 5. The stusdent iimst be permitted to work through the program 

at his own pace again presmnably to reduce the error rate 
by not forcing the student beyond his natural speed of I 
working. I 

Of these five principles the one that appears to be the most I 
valid is the first one* Active, overt responding, however, appears! 
to be more important for learning perceptions and skills than it 1 
does for attaining factual knowledge* I 

The sample page^^ below illustrates a typical early program 
prepared in the light of the above five principles* Note how the 
principles are illustrated in the program. * 

1. A conductor will carry electric current. 

A wire or any substance that will carry 
or conduct an electric current is called 

a * 

conductor. 

2. A copper wire will conduct or carry an 

iilectric current because copper wire is 
a-»good * 

conductor 









i 




A conductor is a substance that will 
carry or __________ an electric current. 

Rubber is not a conductor, so rubber wi^l, 
not an * 

, ; conduct 
^conduct 

"Electric curreist 



An insulator will not conduct an electric 
. Rubber is a good 
'because it- will 



(complete)' 



e 



current 



insulator 
not conduct 
an electric ’ 
current (or) 
not conduct 
current 



Blectric current can flow or travel along 
a > but cannot flow along 

an . 

conductor 

insulator 






William' A. Deterline, 
ionV Englewood Cliffs, N.; J. ; 



An" Introduction to- Programed Instruct 
Prentice Hall, 1962, pp. 17-18. 
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You could receive a "shock” from a copper 
wire unless the copper wire is surrounded 
by an . • > 

* I “r 

An Insulator is a substance or material 
that will » 

(complete) 



A conductor will 

(complete) 



insulator 



not conduct 
electric 
current (or) 
not let . 
current flow 
(or) stop’ 
current 



conduct an 
electric 
current (or) 
carry current 



Criticisms of the five principles and the product as described 
above are not hard to find. The principles have been experiment- 
ally tested and none but the first appear to be essential to learn- 
ing under all conditions of programing. (See Chapter 4). The 
theory of learning based upon operant conditionii]|g and reinforce- 
ment implicit in the principles is open to question on theoretical 
grounds as an adequate model of human learning. Ilany experienced 
teachers felt that the fractionation of the subje^^t matter and the , 
redundancy found in most programs resulting from the principle of 
reducing student errors was contradicted by the siiccess that they 
themselves had in presenting large but meaningful pieces of infor- 
mation to students. 

From the point of view of this chapter, the five principles 
are open to criticism for their lack of heuristic value. They fail 
to illumine the larger problems of teaching. They fail to suggest 
h^ateacher^y writing programed material may develop a more 
sophisticated concept of teaching. No evidence was forthcoming 
that writing programed materials following the five principles made 
a significant impact on the teaching effectiveness of the programer 
as a teacher. In fact, the process of preparing such programs was 
1 escribed as aversive by those engaged in it. 




An inordinate amount of attention has been paid in the past 
to the process of writing frames per se. The drafting of teaching 
frames beebme almost an end in itself, and a complicated lore began 
to develop about how frames should be written. Rather elaborate 
rules have been established to govern such things as prompting, 
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cualng. fading the cues and prompts t practlcalframes , copy frames# 
One of the better books on the subject of frame writing, however' 
is S. Markle*s Good Frames, and Bad.^^ 



As mentioned earlier in the chapter, the Programed Instruction 
Project paid little heed to the established lore of writing teach- 
ing frames, approaching the problem instead from a pragmatic, 
empirical point of view. One programer mentioned his appreciati©n 
that the "very small step approach was not necessary, that fill- 
in-the*-blanks programs are passe." - - Another said that he was 
glad that no rigid format was prescribed. 



• 

Rather than being guided by a set of static principles , "the 
programer focused on what strategies students utilized in attaining 
knowledge, how the content to be learned could best be presented in 
the light of the student’s learning strategies, how the student 
could progress toward the educational objectives with the assistancB 

of the program. 



It has become clear to those working on the project that pro- 
gramed instruction can better be defined by the process through 
which programs were prepared than by programs as an end product . 
Preparing programed instructional materials became a most pain- 
staking and microsopically analytic procedure for doing what most 
c ollege inatructors do extemporaneously in an intuitive way ''««« , 

they develop methods of teaching a course or co^ifider how to delivec* 

a lecture. ^ 
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31 Susan Hankie i Good. Frames and Bad , Hew York; John Wiley ^ 
1964 . ,• . 
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Chapter 8 

IMFORHAL EVALUATION OF PROGRAMING 
AS A MEANS OF ILLUMINATING TEACHING 

INTRODUCTION 



On three occasions the impact of preparing programed materials 
on the teacher was evaluated. The first two evluations occured 
immediately following the summer workshops in programing in 1964 
and in 1965 . The third was a more formal quantitative evaluation 
carried otit in the Winter and Spring of 1966 . The present cKapter 
will review the results of the informal evaluations. Chapter 9 
will contain the report of the formal evaluation. Chapter 8 des- 
cribed the rationale for the preparing programed material and in 
a theoretical way, it s relationship to teaching. 

METHOD 

In March of 1965 twenty or more persons who had prepared pro- 
gramed material under 6LCA sponsorship during the Summer of 1964 
were sent a follow-up questionnaire asking them to evaluate their 
experience in preparing programed material as they looked back on 
it from the vantage point of nine months. A copy of the question- 
Aaire is shown in the Appendix. 

RESULTS 

The first question they were asked is the following: State 
your general evaluation of your programing experience, including 
the field testing and writing. the revisions. The answers to this 
question were quite varied, but fell roughly, into the categories 
shown below with examples in each category. 

1« Programing can be a means to do research on the learning 
process . One respondent mentioned the necessity for doing 
research on the learning process itself. Another suggestec 
that all teaching should be oriented to the learning pro- 
cess of students. 

2* Programing forces systematic thinking about the teaching [ 
process . One programer said that programing was good for 
saying exactly what he wanted to say. Another mentioned 
that programing gave him insight into creating detailed 
nateriala. Still another said that he was forced to think 
not>a ayateAiatlcally about teaching. Another became tho'^ni^ 
roughly perauaded that the material he had worked On was 




















r » progi?ainable t whereas previously he had wondered whether it 
was. Still another reported that he became aware of the 
necessity of careful planning in all areas of teaching. 



3. 



Teachers became more aware of the necessity of formulating 
course objectives . 

One person mentioned that programing helped him set up 
goals for all his teaching; another seemed to become more 
conscious of course objectives. Still another put it this 
wayt "What can be tested can be programed.” Constructing 
a program gave another programer a clear sense of what he 
wanted to accompllshe 



A second question asked of the programers is as follows: 

What new insights or understandings of programing and/or teaching 
oooured to you as result of Katr^nfTnfinni-r»Tir.f^d agd a program?.^ 

The responses of the programers fell into six categories. 

1, The value of inductive teaching (teaching by discovery,). . 

As one programer sald^ "I was reminded tl^at good teaching 
in Inductive- the student is lead to discovery.” 

Another programer mentioned that he discovered in the 
process of programing how little he wantiid to teach ; 
rather he wanted to lead students to diq^overy. 

, t « 

2« Insight into explicit statement of objectives . 

A programer mentioned that programing re'|.nforced the prac- 
tice of stating explicitly the objective^ of the course, 
and the means for carrying them out. Another said, "The 
prerequisite to fruitful discussion is explicit enumeration 
of goals.” A third mentioned that he will plan his course 
in the future more systematically and efficiently with the 
objectives more clearly stated. Finally one of the pro- 
gramers reported that, "Some of the goals I had been work- 
ing toward were worthless!” 

3 , The value of feedback from students . 

g The programer suggested the value of thoughtful student 

evaluation of the materials. Another said, "Field testing 
• showed me the value, of detailed feedback from students.” 

Still another said that programing gave him better under- 
standing of how a student's mind Works - - "1 would not 
have known the difficulties had I not tried to program." 
Still a fourth programer mentioned that he was ^prompted to 
reconsider the contact time that he had with students* 
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4« Student orientation to the learning process * 



A programer mentioned that he had learned that a student 
should answer key questions in the learning process. An- 
other said» "The ^criterion frames struck me - ' a student 
can tell where he stands." 



Problems of communication. 



One programer reported that the difficulty of communica- 
tion of terms etcetera was clearly Identified in the pro- 
cess of programing. Another programer mentioned that he 
discovered problems of communication - - that misinterpre< 
tations often are not detected until the end of a course 
in a lecture or even a discussion. Programing identified 
them earlier. 



A third question was: Looking back over the summer and the 
school year thus far, what were the effects if any, of programing 
on your teaching? ^ 



1. 



Application of programing principles to Other areas of 
teaching. 



A programer mentioned that he was impressqid with the 
possibility of programing adjunctive material. Another 
indicated that he had applied linguistic laws to other 
vocabulary learning in other courses as a result of his 
programing experience. A third programer reported that 
he had revised his lab exercises along lines of the linear 
program. Still another mentioned that he had tried out 
different use of lecture time. A fifth mentioned that, 

"... I am more critical about use of class period and 
testing procedures." Amother mentioned an interesting 
application: In the past he had always lectured by giving 
the rule of generalization first, and then examples of it. 
Now he often lectures by giving examples first and then 
either letting the students discover the rule or giving it 
to them after the example. In addition, he now holds 
quizzes during lab periods before going on to the next 
section. Still another programer reported that he insistet 
that students develop a reading program around a common 
theme and is hoping for good results. 









2, Insights into the learning-teaching process 



■'ii ■' . V. 



A programer said that he is now more patient with the 
learning process after hailing developed a program and tried 
it out on students* Another Indicated that preparing a f 
program had made him awa^e of how concepts can best be put 
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across. St til another programer mentioned that writing 
test frames helped him write exam questions which really 
distinqulsh. 



Question number four read as follows: What difference if any 

have your participation in last Summer's project had on the reac- 
t Ion of your colleagues to programed instruction? 

The answers to this question followed a one» two^ three ratio. 
The lowest numbex^ of responses Indicated a sort of grudging 
consent that was being given by collea gues that px*ogg»amed iii» . 
struction was less bad than had been expected. One program- 
er, for example, said his collea gues were less ready to dis- 
miss programing cursorily. "Some see it as less bacT^LTHT'am^ 
interested." Twice as many responses, however, indicated 
that the colleagues were still indifferent-, had little in- 
terest or negligible interest. The highest category indi- 
cated cautious but favorable response from colleagues. "My I 
part i(hipat ion has stirred a latent interest in othefPs." A 
couple of programers mentioned that there was even consider- 
able interest. One said, "I have been consulted by three 
faculty mtimber 3.'^ Another indicated that i^ the art depart- 
ment other instructors had made use of partd of his program. I 

The final question was: Judging from your { 

what point in one's total teaching practice is programing likely tc 
make an impact? This question served as a summ4)i*y question for t^i 
preceding ones. \ 

By far the greatest number of responses ind|catod that tho 
greater understanding and awareness of the role -played by object*- 
ives in the educational process was the thing t>|iiit struck them 
the most. 



Another category of responses indicated that the teacher de- 
veloped greater appreciation of the learning problems faced by* 
students. 



Another category of responses indicated that there was a 
shift in the teacher's concept of himself a*s a teacher. 



A variety of responses were about specific learnings that 
had occured to them as pregramers, and have been summarized earlier 



e 



Responses that were more cautious and even on a negative side 
were as follows: Programing is not effective as discussion 
for the interpretation of poems • . • I see the possibilities 
and the limitations of programing. One person mentioned that 
the attitude of the instructor toward programing effects the 
• workability of the program. 













Evaluation of Programing Experience by 1965 ProgramTs 

In tha Suminer of 1965 a dozen faculty members of Great Lakes 
colleges prepared programed materials under the sponsorship of the • 
Programed Instruction Project. As in the Summer of 1964, two work- 
shop sessions were set up In which the programers received in- 
struction in the process of writing programs. The description of 
the training workshops may be found in Chapters 1 and 2. 

After each of the two sessions of the 1965 Summer workshops, 
the programers responded to questionnaires with which the workshop 
and the experience of preparing programed materials were evaluated. 
The results of the evaluation of the workshop as such are given in | 
Chapter 2*. . The following section reports the verbatim answers of I 
the programers to the last three questions of the questionnaire 
which deal with the general impact of programing rather than the 
specific evaluation of the workshop. 



The questions are: 



3» Judging from your experience so far, at what points in 

your total teaching pratH^ice is programing likely to make 
an impact? For example, is programing likely to effect 
your setting up objectives for your course, your lecturing 
method or your concept of yourself as a teacher? 

4. What difference, if any, do you think your participation 
in this summer's project will have in t^ip future on your 

colleague's reactions to programed instrliction? 

* 

5. Sv «marize your general evaluation of yot|^|* programing ex- 
peri|ence8 thus far. 



In the following discussion the term "July questionnaire" 
refers to the answers given on the first questionnaire dated July 5 
1965. The term "August questionnaire" refers to the answers given 
on the second questionnaire dated August 9, 1965. Hot every pro- 
gramer responded to the questionnaires. 

Frogramer A 

3* July questionnaire 

I 1 believe that I have assumed clarity of objectives rather 
than actually clarified them for my students. At least, 
.it seems to me that my objectives in some courses have 
< been too general. ‘ I have probably also assumed that my 
students know how to educate themselves-without guidance 
of a very specific nature. Now I am changing my mind on 
this matter. I have also been atro gly motivated toward | 
the lecture method of teaching as a timesaving factor. | 
How«I am beginning to see more clearly the value of stu- I 
dent participation in spite of the time factor. I 

, 316 
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3f August questionnaire 

The first impact in on teaching in general. I am now 
convinced that programing can play a vital part in educa- ‘ 
tion and that it definitely will play that part increas- 
ingly well in the future <• The second impact has been on 
me as a teacher. I now make a more concerted effort to 
establish clear objectives in each of my courses and com- 
municate those objectives to my students. Of course* 1 ha d 
thought that 1 was doing this» but I fear that often the 
objectives were clear only in my own mind . 

4* July questionnaire 

Teachers in departments in general have been anti-program 
as an instructional method. Practically none would agree 
that composition itself can be programed. I think perhaps 
the fact that I am willing to investigate programing and 
to try my hand at it will erase some of these doubts « at 
least among my immediate colleagues » and lead the, way to a 
somewhat more openminded consideration of the possibilities 
of programing on their part. 

4* August questionnaire 



I can only report that my colleagues who have seen my pro 
gram and those who plan to fiei^l-test it are enthusiastic 
about programing for the first time. One colleague may 
stress' programed learning in an Institute which he hopes 
to direct next summer. 

S« July questionnaire 

I find» I am pleased to say* that I have become deeply 
interested in the possibilities which programing offers 
in my field. I have already thought of another subject 
related to my interests which has not yet been programed. 
The work this summer, although it has been taxing, has 
not been a "chore" for me, and I have looked forward to 
each. day* s attempt. I had thought that completing the 
textbook would have been more exciting and more personally 
rewarding; now I would hesitate to say' that. I give most 
of the credit for this interest to the organization of th 



first workshop and to the follow-up procedures: visiting 
editor, outside consultants, provision for test subjects, 
etc. ' 

S« August questionnaire i 

I am now enthusiastic about programing aiid will continue 
to work In this area* I have a second subject for pro- 










graining in mind and hope I can get to it in the near 
future* 

Programer a 

3* July queationnaire 

The experience of programing ia going to make my objectiven 

in all my courses much more specific and precise which 

is to say that programing is a process that develops a 
certain kind of critical sense about the teacher ' s .oblige- f 

.»4itlon: to the student* The experience is going to mea^n a ? 

more critical use of lecture materials! with i clearer I 

distinction between what the student might best learn for I 

himself with suitable guidance and what he can learn best I 

from an organized lectu^ e* I 

4* August questionnaire 
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^ I really don'^t know* I hope they will come to see that 

there is a place for programed instruction becaura I think 
: j it will raise the level at whioh they ca^ approaci*. other 

parts of their teaching! and I will be sl^le to show them 
what they might do! as I would not have ||een able to do 
before. 

5, July questionnaire 

For me it has mednt a new approach to an old problem - - 
- the problem of teaching students how to' pwt words to- 
gether* It has opened up a method of guidance 6f which 
there has been no other form except standing over a studen 
and showing him how to construct his sehtances as he wrote 
them* ’I 

Obviously! then! my evaluation is high* ' 

Programer C 

3« July questionnaire 

I ■ * 

Programing fits in with my conception of teaching. The 
teacher ought to create the conditions wherein learning 
might take place* 






.reaming is a personal appropriation* To facilitate this 
activity teaching should resist the temptation to be per- 
sonal* The personality of the teacher is a threat to leari|< 
I |ng* i,t tempts the student to aocoiiiaodate to the teacher*! 
;^?‘l|i\vpm***onmiity for the sake of basking in his glory, or for 
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the malice of marks of approval. Instead, the teaching sit» 
nation ought to force the student to appropriate the mat- 
erial in his own way. Since the student will try every- 
thing to avoid this, the teacher has to outsmart him. 

4« July questionnaire 

It might cause people to reconsider the cliches they have 
on programming. Everyone has been ready to dismiss my 
project out of hand on the assumption that programing is 
necessarily imcompatible with existentialism. I think 
they do this on the assumption that programing is for 
robots, but not for people. Moreoever, there is the im- 
plication that people are robots some of the time, so 
that it is possible to program robot-like learning but 
not people-like learning. 

I go on the assumption that there is no such thing as ' 
robot-like learning, i.e., there is no evidence that the 
robot is an adequate model for' programed instruction, and 
that the possibility of programed instruction in one area 
opens up the possibility of programed instruction in 
another* 

S. July questionnaire 

Programing is always good experience for teachers. It 
contributes toward clarity in objectives and precision in 
method. 

Programer B 

, 

3* July questionnaire 

I have come to the conclusion that in logic, my teaching 
takes too much for granted on the part of students: I 
have assumed that they would understand without help more 
than I now suspect they do. 

i 

3« August questionnaire 



The answers given on the last questionnaire remain unalter 
4« . July questionnaire 

, , . Difficult to say. The evidence isn't ih. 

4, August questionnaire 
/ I don't knew yet. 
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5* July questionnaire 

Exciting and stimulating* I've learned a great deal* 

5* August questionnaire 

Comments on the last evaluation sheet remain unaltered* 



Programer E 

'3* July questionnaire 

I will probably be more certain to state objectives and 
give complete directions for any student participation* 

' 3« ^ August questionnaire 

It will make me more conscious of the need to give the 
students a clearer idea of the content and objectives be« 
fore beginning work. 

''4* July questionnaire 




I doubt if it will make any difference. 
August questionnaire 






My colleagues are well acquainted with programed instruct- 
ion» at least half of them having written* programs so I 
doubt if my participation will have much" effect. 



S* July questionnaire 



I am still unconvinced of the value of 4 programed course 
of instruction without the personal relation with a teachei* 
and without experimental materials » models or pictures* 









S* August questionnaire 



I'm still not convinced of the value of straight programed} 
instruction in small schools or colleges. I do believe | 
that a combination of good photographs or other audio-visual 
aids with the programed inaitructleia wnMiJ h m 

Programer F 



3« July questionnaire 







ifelther objectives nor method are changed t but eonteait of 
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lectures is moved up a notch if they are based on material 
already covered in programs. Concept of myself as a teach 
er has been influenced. Separation of the teaching and 
learning process - or, at least of the "dishing out" vs* 
"taking in" has resulted in a very strong feeling that 
programed instructions limitations must be kept in mind* 
Programed instruction: work throws into relief certain 
attitudes which I have come to suspect* For example: 
"evaluation" in some respects, is a dangerous preoccupa* 

. tion, I believe. 

3* August questionnaire 

All three to some extent. 



4* July questionnaire 

I haven't thought of this point 
it spreads it attracts interest 

4* August questionnaire 



Can't say. I suppose'as 



1 can't predict this* It is possible thAt the department 
faculty will like the program and this would, I suppose, 
interest them in it. The faculty at large is reasonably 
sympathetic and should be reinforced by the knowledge 
that things are continuing* 

S* July questionnaire 

Every attempt has clarified my view of the problems of 
teaching the material. ‘This has a gene|»ally beneficial 
effect* Hy program work in the past wajS about half bad 
and half good (one useful one not-so-useful program). It I 
is too early to judge this one. 

5* August questionnaire 

I have a much more hopeful view of this summer's project 
after early August and I believe that, if I had the timei» 

I would like to round out a complete set of programs for 
an introductory course but this would take 4 tremendous lo 
of released time* I have about half/df - what is needed , 

. I think* . , . ' .. 






Program er G 



3. July questionnaire 

ft 

Probably mostly my lecturing method, I have grown to appre 
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elate even more than before the advantages of examples 
and case studlss as learning devices for students* 



3* August questionnaire 



It will probably cause me to rely further on the ”iitua- 
tlonlat" or "case study" technique In classes - - to cause 
the student to come to his own conclusions* This would 
be the EORUL principle* I have done this in the past but 
I think I have tended to put the answers In the student's 
mouths more than I should* 



4. July questionnaire 



I think they feel that most of my colleagues presently 
have virtually no Idea of what programing is but that they 
respect the results of programs for that reason* I think 
t'l *'»they will admire the concept of programing whether they 
actually do any or not* 



4* August questionnaire 



This Is a very difficult question to answer* I doubt that 
It will have much impact on my present colleagues because 
most of them in my department seem to have a fixed negatlv 
opinion about programing, feeling especially that it is 
too much ado about very little* 



9 ^ 



In terms of friends and collea gues in other departments. 

I think they have a more positive attitude toward the idea 
of programing* Knowing that I use a program wor*ld not, in 
my. opinion, damage their opinion of the enterprize* 



July questionnaire 



I am generally well satisfied because it has forced me to 
think and write in terms I assumed were either common 
knowledge or were easily understandable* It has frustrated 
me at times because some of my "brain storms" turn out to 
be more storm than brain* But this is exactly why it is a 
beneficial experience* It has also been a learning ex- 
perience and enterprize for me* I find in checking the 
{?cTjMaccurady? of ' some of my tliougHt s.and statements that I have 
beeh careless with my classroom use olF certain concepts 
in the past* The newness of the experience has made it 
. very interesting summer thus far* 
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programing H 

3* July questionnaire 



This experience will have a very definite effect on my 
teachings both in setting objectives and in techniques 
in the classroom. 



3« August questionnaire 
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The two points I expect to be most affected are lecture 
technique and examination writing. There will be a , de- 
finite sharpening of objectives but not a very drastic 
change in content. 

4« July questionnaire 

Mot very much local effect. Both of my colleagues in the 
department here are already very much interested in pro- 
gramed instruction. 

4. August questionnaire 

1 think this will Improve the acceptability of programed 
instruction locally. However, as I mentioned before; the 
climate in the department here is already quite favorable* 
I am hoping to get some people in the department to look 
over the completed program, too, as some of it is closely 
related to their field of interest. 

5* July questionnaire 

This has been a very stimulating thing for me and I am 
very glad that I entered the project. 

S* August questionnaire 



This has been very stimulating to me. The developmental 
testing alone has provided much new "food for thought” on 
the problems of teaching and learning. The difficulties 
encountered by the students trying out my program have 
shown me some large faults in what I had assumed to be 
acceptable teaching methods. These experiences and the 
technical. information, supplied at the two workshops make :: 
me feel tjhat t hav'e learned a great deal which will be ver^ 
useful to me professionally even if I do not develop into 
a good program write^* 
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3* July quostionnalre 

Its strongest impact has been on how I think through a 
problem and» through that effect » onto my sensitivity to 
how students may be working on a problem* l*m less eager 
to "cover” a body of material and more eager to help a 
student hit his stride in discovering the order and the 
nature of the material. 

3. August questionnaire 

Same remarks as in my evaluation of the first workshop. 

4. July questionnaire 

I think the mam thing will be that they'll have a program 
• to look at that is designed with local circumstances in 
mind. Being more concerned with college than any other 
place • they may prick up their ears. (On this campus 1 
n"'*:’ gather' that the chief* notion* About programing is that it 
works for factual material and for average-to-ilow students 
but not for associative material or judgment ^making and 
not for fast students). 

4. . August questionnaire 

Same remarks . . . 

5. July questionnaire 

I've been dismayed at how easy it is to beg the question 
in organizing the presentation of a body of information 
'Cl and delighted to see how students respond to a mode of 
analysis that programming demonstrates very effectively. 

On the one hand I find it hard to be clear and to arrange 
for an "organic" development through a batch of material, 
on the other hand I'm putoff by how slow and repetitious 
have been the three or four published programs I've worked 
on. I'm becoming interested especially in the next step 
after programming - - a step toward identifying students' 
modes of working and then helping them capitalize on their 
mode as well as practice "alien" modes. 

S« . August questionnaire 
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Programcs* J 

3. July questionnaire 

It is likely to be most helpful in the learning of basic 
concepts and skills* I believe I can be more certain of 
accomplishing these kinds of learning through the use of 
programmed material and also cover more material. These 
kinds of materials will not have to be dealt with In lec- 
tures in laboratories and will thus free the teacher for 
"creative” type of teaching. 

4. August questionnaire 

I suppose It will help them to accept programmed Instruct- 
ion* This will depend greatly on the success of the pro- 
gram developed. 

S« July questionnaire j 

My feeling that programing would take considerable thought , 
skill • and time has been reinforced^ but I have not ex- 
periencfd any great frustrations yet. T^ji programming ex-* 
periene| has emphasized the need for opef^|tlonally defined 
objectives; the naed and advantages lnhe|>|nt in intergrat^ 
ing matirlal that the student formerly g|>f"from lectures » 
books •r|abs» etc.; and perhaps also the i4yantages of more 
active j|nd stronger learning situations. 



DISCUSSION 

Testimonial evidence indicates that the programers see the 
following ways whereby programing can apply to broader problems of ; 
teaching: Programing can be a means of doing research on the learn-: 
ing process; programing forces systematic thinking about the teachii|kg 
process; teachers become more avare of the necessity of formulating 
course objectives; the value of teaching by discovery is highlighte<^» 
as well as the value of stating objectives explicitly; programers 
see the value of getting feedback from students and to consider the 
student orientation to the learning process; problems of communica- 
tion were highlighted. 

CONCLUSIONS AND IMPLICATIONS 



There is sufficient testimonial evidence of the value of pre- 
paring programed material for the Insights gained thelte from into 
other aspects of teaching. More systematic study needs to be made 
of the nature and extent of these insights. The following chapter 
reports such a study. 
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Chapter 3 

FORMAL EVALUATION OF PROGRAMING AS A MEANS OF 
ILLUMINATING TEACHING 




INTRODUCTION 

In Chapter 7 we described the process of training programers 
and of preparing programed instructional materials* Programers 
were trained to formulate objectives in observable • testable terms, 
to write criterion test items that validly and reliably assess the 
students progress toward the objectives, to select the contents of 
'the program in terms of the objectives, etc * From the very first 
training workshop many programers perceived relationships between 
the principles and techniques of preparing programed materials on 
the one hand and many broader educational procedures used by coll- 
ege teachers on the other* Chapter 6 records some of the insights 
that programers gained into the relationship between programing 
and teaching* 

Such test|.monial evidence ae recorded in Chapter 8 may be 
sufficient' to ccpvlnce many persons that preparipp programed ma- | 
terials is a process that clarifies many other phases of teaching 
not directly riiikted to programing* However, te||itiionial evidence ^ 
is subject to j|||ny well known criticisms most im]^|ptant of which I 
is that there 'ii no control group with whose statements the test- ! 
imonies of the programers can be compared* Without a comparable ( 
control group it is impossible to tell whether the insights ex- [ 
pressed by programers are any different from insights that any i 
other college teachers might express about teaching* 

Statement of the Problem 

If the preparation of programed instructional materials is to 
. serve in the future as a device for preservice or inservice educa- - 

I tion for teachers it seems desirable to establish its relationship 
to other selected aspects of teaching on as firm a research base 
as possible* That is, the positive benefits of preparing program- 
ed materials needs to have something more sqlid than testimonial 
evidence as its foundation* 

The general problem with which this study was concerned is 
that of exploring the relationship between selected aspects of an 
instructor's concept of teaching and the principles and techniques 
he learned in the process of preparing programed instructional 
materials* In more general terms the problem is that of finding 



* Written in -collaboration with Clerence /^uba 
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out the changes which develop in one's theory of teaching out of 
the process of programing. Some of the specific questions dealing 
with this problem are as follows: 

1. Does preparing programed materials influence one's con- 
cept of teaching. 

2. Is it possible to formulate assumptions and questions 
about selected aspects of teaching which Vand questions) 
differentiate programers from non-programers? 

• 

3. Using the assumptions as criteria can Inaependent judges 
reliably differentiate statements about selected aspects 
of teaching made by programers from statements made by 
non-programers? 



4. Specif ically • what assumptions and questions differenti- 
ate a programer from a non-programer and which assumptioii|8 
and questions held by a non-programer differentiate a 
non-programer from a programer? 



Assumptions abopt Selected Aspects of Teaching 

The follc^^ing critical assumptions about t|ie selected aspects 
of teaching wep| utilized as guides both in the ^instruction of 
questions that^pade up the interview used in thi|^!ptudy» and in 
constructing '^he criteria whereby the judges ratq^;,the answers 
given by the respondents in the interviews. (See Appendix F-1 for 
a copy of the interview schedule) The statements of the assumption 
given below are not necessarily systematic, definitive or polished. 
They are presented here, however, verbatim and unedited so that 
the reader can judge for himself the adequacy of these assumptions 
as they were actually used in the study: 
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1. In preparing programed instructional materials the 

methods of instruction have more value relative to con- 
tent than is usually ascribed to them by college instruc- 
tors. The importance of teaching methods as compared 
with content increases in the estimation of the instruc- 
tor who has prepared programed material i he pays increas- 
ing attention to and shows increased interest in teach- 
ing methods. (Used as basis for interview question two.) 



2. Formulating objectives in explicity, behavioral, objec- 
tive, operational terms (in terms of outcomes of student 
behaviors) is a cardinal principle of programing. After 
having been trained in the principles and practices of 
programing and having prepared programed material, the 
programers will move in the direction of expressing a 
need to formulate general teaching objectives as des- 
cribed above. (Used as basis for interview question four 
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An impoirtant principle of programing is that students 
should be tested in terms of progress in attaining the ob- 
jectives of the program (this is the ranking function of 
tests). After having been trained in the principles of ! 
programing and having prepared programed materials • the I 
programer will move in the direction of developing tests I 
and measuring student progress in relationship to teach- 
ing objectives. (Used as basis for interview question 
five. ) * 

Analysis of materials to be programed is related to and I 
grows out of the objectives of the program^ and material I 
to be programed is to be organized empirically and prag- I 
matically in terms of the students ability to deal with I 
it. The programer is to be guided by feedback from the 
students; the programer is not to assume that students see 
all the relationships, meanings, etc . , but will check to 
see if they actually do; the programer will be concerned 
about the amount of information presented in each step of 
the program, the relationship of one step to the other, 
and the relationship of concepts and theories within his 
program* A programer will relate objectives, get feed- 
bAok from students, and relate materials within a course* 
(Used qs basis for interview question si^*) 







5* As a rqsult of having been trained in t)if(i principles and 
techniques of programing and having pregii|>ed a program, 
the programer will be more inclined to students' 

specific knowledge of the material to be>taught prior to 
the students' starting a course and to reorganize the con- 
tent and methods of a course in the light of what he dis- 
covers about students' previous knowledge. (Used as basis 
for interview question seven.) 



6. Students need to be actively and responsively engaged in 
learning; their activity improves not only their learning, 
but also provides feedback to the programer about the 
strengths and weaknessi^s of his program. A programer, 
therefore, will realize the importance of developing an 
open searching atmosphere in the classroom, encouraging 
active participation in learning and discovery, and a 
questioning attitude on the part of the students. (Used 
as basis for interview question eight.) 



7* The material itself must be made intrinsically motivating, 
since the programer cannot count mainly on interpersonal 
relationships with students to motivate them* Motivation 
in programing comes from the challenge of discovery, the 
freshness of the approach to the material, the encourage- 
ment that comes from successfully solving problems* After 
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having been trained in the principles of programing and 
having prepared a program • the programer will move in the 
direction of motivating students by the way in which he 
presents the material. (Used as basis for interview 
question nine. ) 

The purpose of testing students is not only to evaluate 
the progress of students toward the objectives of the pro* 
gram» but also to give the teacher data with which to re- 
vise his assumptions about students' knowledge » th.e level 
or difficulty of"the content* and the sequence of the 
material (this is the feedback function of testing used 
in the evaluation of teaching). Testing should be fre- 
quent* and the results immediately fed back to students. 
After having been trained in the principles and techniques 
of programing* the programer will move iq the direction of 
incorporating the above principles in hiq regular teachinc 
procedures. (Used as basis for Interview question ten.) 

*5^ 

After leaving been trained in programing pi:|lnciples and 
having'* prepared a program* the programer ^^wiH be 1) more | 
to ufi now inatruetional media Iq hie teiohing* 2) 
favor(|j|)ly disposed toward their use* 3) ;|n;pow how to use 
them lliscriminatingly * and 4) Le inform^||‘ about them. 
(Used'^as basis for interview question llvl 

To summarize* it is assumed that programers will do the 
followingi (The summary was used as a basis for interview 
questions one* three and 12) 

1) Pay increasingly more attention to methods of teach- 
ing. 

2) Formulate objectives explicitly in operational be- 
* havioral terms. 

3) Evaluate students' progress with greater regard to 
course objectives. 



4) When organizing and analyzing course content* will do 
so in relationship to objectives* will get feedback on 
adequacy of organization from students* will not as** 
sume that students understand material* but will find 
out empirically. 

5) Find out what students know about specific material in 
a course before teaching it to them. 

'6) Get active participation from the student in learning* 
develop an open searching classroom atmosphere* and 

obtaining feedback from them. 
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7) Develop materials and content that motivate students 
• because of the challenge of the discovery they ccn- 

talnt because of the motivation they get through 
successfully solving problems, through the freshness 
of the approach, and the pacing of the material. 

8) Assess students frequently not only to evaluate their 
progress toward the objectives, but also to find out 
wH^Q^e the course needs improvement. 

9) Develop favorable attitude toward, add more frequent 
, ‘use of, new Instructional media. 



METHOD 

The following procedures were used in conducting the study: 
Subjects 



A total of 24 subjects were used in the stu^^. Twelve GLCA 
instructors who .iirpre trained and who prepared progipfmed instruction* 
kl materials unde|? the ^egis of the Project during' the summer of 
1L965 and twelve iiilembers of a control group. The j^nogramers will be 
ailed the experl|iental group. r - 






f 
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Ideally, th# experimental group and the control group would 
ave been randomif selected from a common pool of persons who appl- 
‘led to prepare programed materials. Since, however, such a proce- 
ure was manifestly impossible to follow under the conditions of the 
Project, the next best. step was to ask the academic dean to select 
ersons comparable to the experimental group to serve as a control. 



In forming the control group of twelve members the Director 
isked the academic dean of each college from' which the 19S5 pro- 
Sramers came to match each 1965 programer with a colleague who woul^ 
lerve as his control. The Director provided each dean with a list 
yf criteria for the selection of control faculty members. The 
sriteria were as follows: 

1. The control person should come from the same general 
academic area as the programer; for example, from natural 
science, humanities, modern foreign language, or social 
science. He need not, however, necessarily be from the 
same* department. 

2. the obntrol should have approximately the same general 
level of ihterest in teaching and similar willingness or 
Unwillingness to try out now educational methoHs. 
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3* The control should have approximately the same general 
level of teaching ability* 

4* The control should have approximately comparable expert 
ience as a college teacher. 

S* The control should have no experience in preparing pro- 
gramed instructional materials. 

Research or counseling skills • however • were specifically ex*^ 
eluded as criteria for the selection of the control group* * Selec- 
tion was to be made entirely on the basis of teaching skills and 
interests* 

Procedures 



Ideally a research design using pre and post measurements ant 
experimental and control groups would have been employed* Althougli 
experimental and control groups were used« various circumstances 
prevented the use of pre-measurements. The most important circum- 
stance was a lack of a set of assumptions about teaching prior to 
the beginning of the summer of 1965« FurthermorfJt an adequate i**" I 
terview instrume^nt for obtaining the statements of programers and | 
non-programers ifith respect to the assumption was lacking* i 

•* 

For purposes of this research it was assu|i.qf| that prior to 
the summer of |j|65 both programers and controls comparable 

conceptions of^'teaching and that any dif ferences-^hat were to be 
found between their concepts of teaching following the summer of | 
1965 were due to the training that the programers received and the! 
experiences they had in preparing programed instructional materials 
during the summer of 1965* I 

j Variables I 

It is assumed that the independent variable in this research 
was the training and the experience of preparing programed mater- 
ials and field testing them as described in Chapters 2 and 7* How- 
ever» in as complex a situation as this project presents it was 
impossible to control or even recognize all the variables that 
might have been operating. The training workshops consisted of twe 
; three-day workshops held three weeks apart » the first in June and 
the second in July* and the additional seven weeks of programing 
which for most of the programers extended into the fall of 1965* 

i lt should be noted that there was a iriinimum of discussion about th 
relationship betVeen programed instruction and teaching during the 
summer of 1965 • Mo systematic effort was made to help the pro- 
gramers perceive what these relationships were* This experience 
of preparing programed materials differs from the assumed random 
activities of the control group during the summer of 1965* 
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The set of statements made by programers & J non-^programers 
in an hour-and~a-half long interview conducted by Dr. Clarence 
Leuba» former chairman » Department of Psychology, Antioch College 
served as the dependent varialSle. The interview schedule is given 
in Appendix F-1. The interviewer did not know until the end of thi 
interview who was an experimental and who was a control subject, 
except in two instances where he was familiar with the campus sit- 
uation. 



Method of Collecting Data--Interview Procedures 



After the deans had notified the Director of their selection 
of control persons the dean was asked to write a letter to each 
prospective control group member asking him whether he would be 
willing to be interviewed by Dr. Clarence Leuba on ^'some selected 
aspects of the total process of teaching as a part of the overall 
Programed Instruction Project”. The controls as well as the ex- 
perimental group were paid $25.00 each for being interviewed. Af- 
ter they received the letter from the dean and all had indicated 
their willingneps to be interviewed Dr. Leuba made appointments 
with them to caipry out the interviews. These wet's conducted from 
six to nine lontbe after the workshops during the months of Janu- 
ary, February, and March, 1966. 



The interviewer, after making the necessary appointments, 
spent a day or two on each campus interviewing everyone who was 
either an experimental or control person. The interviewer tape re 
corded each of the interviews and also wrote down||' the pertinent as- 
pects of the responses verbatim. When the interviewee had finishe 
his answer to a question the interviewer read what he had written 
to the interviewee who then agreed to what had been written or 
made additions or changes to it* 



In addition to the twelve questions on the interview schedule 
the interviewer also asked a thirteenth question aimed directly at 
the programing background of the respondents and at their opinions 
regarding the degree of applicability of programing principles and 
procedures to the development .of other educational materials. The 
responses to this question were treated separately from the res- 
ponses to the first twelve questions. 






An interview procedure was selected in favor of a question- | 
naire approach’ because ‘it was deemed necessary to allow the respon 
dents as much latitude as possible in giving their answers to the 
questions in the interview schedule. Multiple choice questions or 
a check list would have been unduly restricting. To have the sub- 
jects, however, write out their answers to a questionnaire would 
have precluded further questioning for purposes of clarification. 
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■t^rocessing the Interviews 



The interviewer conducted a total of 27 interviews: 12 experi 
mental^ 12 control^ and 3 practice interviews* The responses to the 
interview questions as recorded verbatim by the interviewer were 
sent to the project office where the answer of each respondent to 
each question was typed on a separate page of which multiple copies 
were made* In its final typed form each interview consisted of 
twelve pages, each page containing one of the twelve questions of 
[the interview and the answer given by the respondent* Each respefn- 
jdent was assigned a code number* His code number was stamped on the 
;back inside margin of each of the twelve sheets on which his answers 
jwere typed* The code number was so placed that when the answer 
isheets were punched and placed in a binder the code number was com** 
pletely hidden underneath the binder* 



The answer sheets of the respondents in their final typed form 
|Were disassembled ~and sorted into twelve piles, each pile containing 
the answers to one of the interview questions* Pile number one con** 
tained all of the answer sheets from the twenty four respondents to 
question number one| pile number two contained the twenty four an- 
wers to question number two, etc * Tha answer pages to '..each ques- 
ion were then shuffled into random order, reassembled, 
lone set of answers in each binder* The judges who werq 
interviews were prevented by the random ordering of thq 
rom falling inadvertently into a rhythm or set pattern 
ithe .answers * In summary, at the time the judges rated 



and bound 
to rate the 
qnswer pages 
Of rating 
t)»e 24 inter-4 



d 



, views they were contained within twelve binders, each binder con- 
itaiping twenty-four randomized sheets on which were typed the answer 
of the experimental and control groups to the twelve questions that 
)»ero asked in the interview* 

bating the Respondents Answers 

Three judges were selected to rate the responses of the inter- 
viewees to the interview questions* The judges were Dr* Leslie 
Beach, Department of Psychology, Hope College; Mr* James Brink and 
Hr* Roger Scott, Graduate Assistants, Center for Research on Learn- 
ing and Teaching, University of Michigan* Each of the judges, 
familiar with principles of programing and experienced in teaching. 
Was given two full days of training by the Director* The first day 
Of training consisted of going over the assumptions that were to 
ser^e as the criteria for rating the answers to the interview ques- 
]tions* At that time, the final rating scale was described and made 
final* One of Dr* 'Leuba*s practice interviews was used to illus- ■ 

trate how the material was to be rated* 

1 

; • ... 

In the sedond training session, the three judges were brought 
itogether and were given practice in rating the remaining two prac- 
ice interviews. The attempt was made to get as high agreement 
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among the judges as possible during this training session. When- 
ever disagreements occured the reasons for the disagreements were 
identified and clarifications were made in the procedures for rat- 
ing the answers. 

The interviews were rated in two stages. The first stage 
consisted of a sentenee-by-sentence , phrase-by-phrase, and word-by* 
word analysis of the answer to a given question in terms of the ex- 
tent to which the answer agreed with or ran counter to the princi- 
ples of programing expressed in the assumption underlying the ques- 
tion under consideration. Tha judges were instructed to encircle 
key phrases, to mark words, underline sentences, use pluses .and ' 
minuses and numbers ranging from +3 to -3 to give a first rough ra- 
ting to the respondents answer. As a guide to their rough analysis 
they were asked to use the seven point scale given below: 



-3 


- 2 
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+ 1 


+ 2 


+ 3 




General- 


Specific 


General- 


Irrelevant 


General- 


Specific 


Genera 
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ization 


Operat- 


ities , 




ities , 


Operat- 
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ional 


no "why” 




no "why" 


ional 


plus 




examples 


examples 








examples 


examp: 
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If, for axampls, a respondent's statement, was irrelevant to 
the assumption it was encircled and marked zero; if- an answer con- 
tained a specific operational example of the assqmption the example 
was circled or underlined and marked plus two. the first 

stage of was completed the judge had annot||ed, underscored) 

and sprinkled fhe answer sheet with negative and positive numbers. 

The second stage of judging the interviews consisted of giv- 
ing a final rating to the respondent's answers. The judges were 
given the following instructions: 

I After you have marked the statements, sentences, and words 

I in the answer book according to the seven point scale above 

combine your scores and give a final global rating to the 
answer on the five point scale below, using the following, 
rating: 

Rating : In the light of the evidence found in the answer to 
what extent does the respondent show that he is aware of the 
assumptlont principles, and techniques of programing that are 
relc^vant to the situation described in the question? 
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Evidence of 
lack of aware* 
ness of 
principles 



Little or no 
awareness/bal- 
ances between 
awareness and 
lack of it 



Some aware* 
ness of 
principles 



Consider- 
able a- 
wareness 
of princi* 
pies 



Thorough 
and ex- 
plicit a- 
wareness o 
principles 



No formalized rule was used to combine the pluses and minuses 
and numbers from the first stage in making the final global rating. 
The first stage served as a device to insure careful reading by the 
judge rather than as a process of unitizing or systematically 
weighting the various parts of the respondents* answers. 

The judges ware asked to rate all the twenty four respondents!' 
answers to one question in one sitting. After completing the first] 
stage they were to make a tentative final ra*ting to each of the 



answers and then to review and confirm their 
wrote their final rating from the range of 1 
the answer sheets in the answer booklets and 
as confirmation. The judges completed their 
weeks . 



final rating. They 
through 5 on each of 
initialed their rating] 
rating in about three 



to the Dlrec- 

reassembled 



The judgee returned the twelve answer booklets 
tor with their ratings to each answer. The Director 
the interview questions so that the answers to al| twelve questions 
by one respondeift were put back into one folder. ^The judges rating) 
were then tabulated by judgev question} and respondent. (See 
Appendix F-2) ‘ 

RESULTS 



I The statistically significant 

among the judges as determined by Kendall's 
Idance, 



amount 

t 



of agreement was found 
coefficient of concor- 

W,« 0.908. Average intercorrelation among judges R= 0.862 
l(p<.001). (See Appendix F-3.> 

s 

f The difference between programers and non-programers on the 

^ test was found to be significant (p^.05). (See Appendix F-4). 
pifference between programers and uninformal controls* was signifi- 
fcafit (p<o01) as determined by the Mann-Whitney test. (See Append!: 
F-5). 




*Dr. Leuben discovered that half the control group was rela- 
tively well informed about programed instruction, the other 
half was not. For purposes of some analyses the former grou] 
is called the informed controls (Cl) and the latter group is 
called (CU). See section in "Results from Qualitatives Eval- 
uation!^* 
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The point blserial correlation between matched pairs, one 
programer and one control person, rpb= 0* 566 (pi(£.05). Since the 
£ priori assumption was that programers would be superior, the 
d^ifference is significant. (See Appendix F-6). 



The interview questions were rank-ordered on the basis of 
their power to discriminate between programers and non-programers . 
See Appendix F-7 for the statistical procedures for finding the 
ranks. Below are the interview questions by number, the assuroptlon|s 
by number on which questions are based and which the judges used in 
ranking the respondents' answers, the rank order of each question 
in terms of Its dlscrimlnabillty , power to differentiate programers 
from non-programers , and a summary statement of the question. The 
significance of the differences of the ranks of each question was 
not obtained. * 



Interview 

Question 



Assumption Rank order Summary of Question 



11 

2 

12 



9 

1 

10 



1 

2 

3 



6 

3 



10 

6 



5 

6 



1 

9 

4 



10 

7 

2 



7 

8 
9 



5 

10 



3 

8 



10 

11 



12 



Use of new media 
Content vs. method 
Recent changes In 
thinking 

Analysis and organ- 
isation of course 
Role of the teacher 
T|iSoher-student rela 
t|dnship 

Neg| steps In devel- 
opment of teaching 
Sources of student 
motivation 
Formulation of ob- 
jectives 

Student evaluation 
Frequency of evalua- 
tion 

Evaluating students 
previous knowledge 



Results from Qualitative Evaluation 



The interviewer collected data from which further qualitative 
evaluation of the impact of programing can be made in addition to 
the results described above and in Chapter 8. He did this by askin^E 
a thirteenth question, the answer to which was not recorded on the 
tape recorder nor given to the judges to rate along with the other 
twelve* Until he heard the answer to the last question, number 13, 
the interviewer was not aware of who was the control and who was 
the experimental person, except in a couple of cases where he knew 
the campus situation. This question was aimed directlv at the 
programing background of_J:_he r e8Pondaiit8_.a nd „at_theJjp-^pi^i^^^ 



2 d 6 



' 



i 









'TV'S 









Z " • =* V 'll* 












o 

• ERIC 



gardlng the degree of applicability of programing principles and 
procedures to the development of other educational materials* 

Question 13 had three parts, (a), (b), and (c) as follows: 
Part (a) Do you happen to be familiar with the methods 
used in the development of programed instruction? To 
this part, the 12 programers all obviously answered 
'•yes”. 

Part (b) If yes, do you feel that the • principles and 
techniques involved in the preparation of programed 
instruction have - or do not have - application to 
other methods of instruction? In other words, are 
the principles and techniques of programing appli- 
cable only to preparing programs or do they have 
broader applicability in education? All the programers 
without exception, saw broad applicability in educa- 
tion! and in most cases, they were very emphatic and 
certain about this; the answers came quickly and with- 
out hesitation* Typical answers were: "In all forms 
of teaching”; "Extremely useful" "Hide (or broad) 
applicability"; "They - the principles of programing - 
are based on sound psychological (or educational) 
principles"; I became more aware of their |eneral 
appicabllity" * 



Part (c) Just how are they applicable? 
specific methods of teaching? 



To what 






The following were the most commonly mentioned applications; 
with the exception of the last one, each was mentioned by half or 
more of the respondents* 

1* Getting regular and frequent feedback from students re- 
garding what is difficult or unclear, checking on the assumptions 
and presuppositions they as teachers had been making regarding what 
students already know, what interpretations students make, what stu 
dents understand or do not understand, how students learn, what 
iconclusions students are drawing as from lectures or reading; and 
{revising educational materials in the light of the informal ques- 
Itioning of students and of formal testing* It was not until they 
|had gone through the rigorous and repeated testing of their ed- 
ucational materials by getting students' reactions as required in 
the preparation of programed material, that most Instructors began 
to fully realize the importance of this frequent and regular feed- 
back from students* 

2* Not skipping any esf.entlal dteps in problem solving and ii 
:he development of knowledge in general; proceeding by steps of 
appropriate size and in the proper sequence, no matter whether the 
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overall approach was a chronological* historical* or purely 
logical one. 



3. Stating objectives in clear* specific* objective* be- 
havioral termss and not in just vague* general ones* like "a better 
appreciation by students of my area"; stating objectives in such 
a way that students can be tested regarding the extent to which 
the objectives have been achieved. 



4. Being more critical and more precise in the development 
and use of instructional materials; requiring more discipline* 
thoroughness* and clarity from oneself and being more careful in 
the or|;anizatlon of educational presentations; thinking about how 
one teaches as well as about the content of one's teaching. 



5. Enabling students to find out at once whether their - the 
students' - knowledge was correct or Incorrect and whether their 
understandings were adequate or not. This Implies the necessity 
;of frequently questioning and testing student knowledge and under- 
standings; and giving them immediately the results of these eval- 
juatlons; this motivates learning and prevents students from por- 
osis ting in error* 



It is worth noting that none of the 12 progrqmers mentioned 
one of the procedures previously considered basic in the develop- 
ment of programed instruction; namely* reinforcing the student 
yith the correct answer. The 6LCA programers appi^rently did not 
‘feel that giving correct answers indefinitely woul4 continue to 
'furnish students;^ with adequate reinforcement. In fact* they seemed 
,to consider such a procedure boring and an unfortunate aspect of 
much current programed instruction at the college level. What 
apparently did impress the workshop participants favorably was the 
limportance of (1) finding out precisely what students didn't know 
tor didn't understand and (2) of quickly bringing this lack to the 
attention of students. VIhat was undesirable was not necessarily 
Istudent errors* but the Instructor's failure to know about them 
pnd to use them either to motivate learning* or to revise educa- 
tional materials. 






j The thirteenth question uncovered an interesting finding about 
the control group. There had been such effective publicizing of 
programed Instruction among the 6LCA colleges that* in several of 
them* the members of the control group turned out to be less naive 
regarding programing than had been hoped. Though none of the mem- 
pers of the control group had attended the 6LCA summer workshop 
on programing or had experience with the development of programs 
of their own* half of them indicated* in-answering the last inter- 
view question* that they were either somewhat or quite familiar 
with programing; some had even used programed instructional mater- 
ials. The other half was only slightly* vaguely* or not at all 



2 38 




































1-1 



I 






i 



n 






informed. The control group was consequently divided into "the 
Informed Controls (C-I) and the Uninformed Controls (C-U) each 
containing six subjects. (See Appendix F-2.) 

Like the programers, the informed members of the control grouj 
were unanimous in their opinion that the principles and techniques 
involved in the preparation of programed instruction had applica- 
tions to other methods of instruction and generally broad applies- ^ 
bility in education. They were less emphatic and definite* however; 
and usually did not see as many applications. They made „ 

statements as "My impression is that there is general applic.ability 
"I see some possibilities of applications"; "I'm not sure* but 
think there are applications". 

As might be expected* the six relatively uninformed members 
of the control group failed to see any applications. 




DISCUSSION 




The statistical analysis of the judges ratings of the inter- 
view data and the testimonials from the programers themselves com- 
pared with the controls strongly suggests the conclusion that pre- 
iparlng programed materials does influence one's concept of teaching 
(in the direction outlined in the assumptions. This is a change n j 
leducational ideas and teaching behavior in so far as talking is 
behavioral. 

The 18 faculty members (the 12 members of the summer workshop 
and the six Informed Controls) were unanimous in their opinions 
that the principles and techniques of programing are applicable and 
useful in many aspects of college teaching. The 12 respondents who 
had taken part in the summer workshop* and had actual experience in 
the development of programs of their own* were more emphatic and 
certain in this opinion and seemed to see applications more clearly 
specifically and extensively. In short, the closer one is to pro- 
'graming the more clearly he sees carryover to other areas or 
teaching. The first question in the section on the Statement of 
the Problem" can therefore be answered in the affirmative. 



i 



, 



1 



The statistical analysis indicates that the three judges agree 
on their rating of the statements made by respondents in answer to 
the questions of the interview. Thus* the method of interview and 
natinp r the interviews is a reliable method* and the third question 
- iudgi^w yaliablv rate q^gwejis under the section, "Statement 

of fhe "PPOblibm" cari’~’Ee answered in the affirmative. 






It is now possible to answer the second question raised in the 
itatement of the problem in the affirmative. That is* it was possi 
»le to formulate a number of assumptions and from them ask question) 
ibout teaching* The answers to the questions when rated by in- 
lependent judges uplng the assumptions distinguish programers from 
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non~programers. The content of the assumptions defines the way in 
which the concept of teaching of instructors who have prepared 
programed materials differs from the concept of instructors who 
have not prepared programed materials. Faculty members who have 
had experience in the development of programed material incorpor- 
ate more of the assumptions into their talking about teaching than 
do those who have not. Faculty members who have had experience 
in the development of programed material feel that such experience 
^ ^ has a broad educational applicability and the expression of thei|i 
general educational thinking shows the impact of this experience. 
Thus, we can conclude that the preparation of programed Instructloi^- 
al material does Indeed seem to have a broad • educational Impact. 

In addition to the significant differences found between the 
programers and thelr> controls • th^ significantly high point biser- 
ial correl ation c oefficient ihdica'tenB that there is an orderliness^ 

It can belg^ifri^ that good teachers in the programing group were” 
matched by tfie^eans with good teachers in the controls » and poor 
ones with their counterparts. The difference between the groups 
was probably not due to their Initial teaching abllltya but they 
gained Insight about teaching processes as a function of being 
Involved in programing. 

In the ’’Statement of the Problem" the question was raised of 
which assumptions and interview questions served ^9 differentiate 
most clearly the programers from non^'programers. ’ifhe question can 
|be answered. Interview questions !!• 2» 12| and | (based re- 
|spectlvely on aspumptlons 9« 1» 10 • and 4) most c|early differen- 
itiated programers from non-programers . In accord ^Ith the flndlngc 
of Lysaught reported earlier in Chapter 7, the queitlon dealing 
jwith the use of new media ranked number one. The preparation of 
Iprogramed material favorably disposed the programer relative 
to the non-programers, to the use of new media; and perhaps incline 
him to use them, to use them discriminatingly, and to be informed 
about them. 



Second in order of discriminating power was the question of 
the importance of method versus content in teaching. Programers 
placed a greatex^ value on teaching method relative to content than 
did non-programers. Third in rank in disciminating power was the 
question 12 asking for recent changes in their thinking. The 
answers of the programers indicated that many of the answers given 
to the earlier questions in the interview represented recent 
jehanges in their thinking. This does not mean, of course, that 
the control group was not thinking about teaching nor that the 
thinking of its members was not in the process of changing, but 
rather that thinking of the non-programers along the lines out- 
lined in the aseumptiotis and questions had not changed as sig- 
nificantly as had that of the programers. 
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The question that stood fourth in rank in discriminating poved 
was number six, dealing with the analysis 

course content. Although the assumption on which this <1"*®^*-°" “ 
based is complex, the interpretation can tentatively be made that 
the analysis of the course content in terms of previously stated 
obiectivee and the empirical testing of the sequence of the con- 
tent on students in order to obtain feedback 

major learnings obtained by the programers in contrast to the 
jnon-programers e 

i It is noteworthy that interview question numberfour dealing 

with the importance of formulating objectives explicitly, be- 
•haviorally, ^objectively and in operational terms was ranked only 
'ininth in its power to discriminate between programers, "°“- 

Iprogramers in view of the low rank given to the question about 
objectives, particularly in comparison to the relatively high rank 
given to the question six about the analysis and organization of 
bourse material, it becomes necessary to modify ® .f ““^ns 
in Chapter 7, in which it was asserted that, • • • i* still remains 
true that no step in the entire process of writing P*‘°8ramed 
material made a greater impact upon the programer s understanding 
of teaching than learning how to formulate 

objectives." The data from the study indicate that *be analysis 
and empirical organization of the course material as required by 
programing made a greater impact on the 

after programing than did the formulation of obj^c i . 
higher value of obtaining feedback in organizing .the content ^®" . 

lative to the value of formulating objectives tencjs to be emphasized 
as well in the -answers to question 13 reported ip the section on j, 

resultSs ; ' 

It is also noteworthy that on question number 7, dealing with 
the evaluation of students* previous knowledge, the non-programers 
responStSmore il line with ?he objectives than did the programers 

A word of caution should be addeds Sheldon C. Reed, speaking 
of research on genetic or environmental effect upon intelligence 
states, '*...we should perhaps think of our data as yeing more 

erroneous than our speculation, if on© may f ^ ^ 

orthdox idea. Clearly, the worst e^'^or possible in this area of 
tudy is to be certain of our data.'*^' Such sentiments also apply 

to the study reported herein. 

A second word of caution should be added. Doe© programing 
procedure produce actual changes in the classroom teaching rethod? 



27 Sheldon Ce Reed, **The Evolution of Human Intelligence: Some 
Reasons Why It Should Be a Continuous Process**, Am. Sc. Sept- 



ember, 1B65, B. ijj21i 
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We will venture the opinion that the long run impact on the 
actual education procedures and classroom behavior of most of these 
faculty members may be slight or even negligible; at least unless 
there is systematic following by way of encouragement and guidance 
and freed time for the actual application of the educational pro- 
cedures and principles which these faculty members now feel can 
contribute so much to more effective college teaching* Long es- 
tablished educational ideas and practices are not necessarily 
effectively changed by a few months of practice and a couple of 
workshops useful as these may be* One of the main weaknesses - 
often a fatal one - in programs for educational innovation is a 
failure to follow through over a long enouf.h period of time** 

It remains to be seen just how much of this very time con- 
suming experience with the actual development of programed in- 
struction of one's own is really necessary for the carry over or 
transfer of the more useful programing principles and technic^ues 
to other types of instruction* Maybe, practice in the preparation 
of parts of a few sample programs might suffice, and the time thus 
saved could be used in the application of programing principles 
to the instructional methods moatL_used by an instructor, such as 
in the prepar^ation of his lectures, syllabi, or examinations* 



CONCLUSIONS AND IMPLICATIONS 

This pilot study strongly suggests that preparing programed 
materials holds promise as a significant procedure for analysing 
and improving some important aspects of teaching, especially those 
having to do with the predesigning of learning conditions* But 
the door has just been opened* Many large areas of the total 
teaching-learning process remain to be investigated and developed 
via the preparation of programed instructional materials* 

Further sustained and more rigorously controlled research 
needs to be conducted to ascertain which aspects of teaching are 
most influenced by programing and to ascertain the point of 
diminishing returns, that is, the* point at which further programing! 
fails to produce new insights into teaching* What needs to be done! 
to help teachers translate and generalize their new insights to 
their actulil classroom behavior is still open for further inves- 
tigation* Such research will help determine the future role of 
preparing programed instructional materials in the total pre- 
service and in-service education of teachers* 
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CHAPTER 10 



SUMMARY, CONCLUSIONS AND IMPLICATIONS FOR 
FURTHER RESEARCH AND DEVELOPMENT 

Review of Purposae of the Project 

Thtt purposes of the Project as outlined in the preface are 
jgrouped below into five categories: 

i 1. To develop programed instructional materials. 
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2e To evaluate and compare uses of programed instructional 
materials prepared in 1964 and thereby to discover where 
programed ihstruction fits into the curriculume 

3. To evaluate existing commercial programs. 

4V To promote basic research in instruction; to evaluate the 
broader effects of programed instruction; to compare in- 
dividual versus team preparation of programed instruction- 
al material, and to ascertain if evaluating programed 
Instructional material has greater effect on instructional 
processes than does preparing programed Instructional 
materials. 



5. Dissemination of results. 



i Summary of. Project Activities 



y I 



I All_ GLCA colleges participated in the three-year Programed ■''V, \- 

jlnstructioh Project and were kept in touch with the various activi- 
jties of the Project through the Director who communicated with a 
iLiaison person on each campus. The Liaison Committee met a total | 

|of five times. The Director described the Project to the facultie b 
!of all the colleges and made numerous contacts with facultyn members^ 
on the various campuses for a variety of purposes related to the 
project. Two major all-Association conferences were held. The 
first was a series of winter work conferences held early in the 
life of the Project to introduce the colleges to Programed Instruct)* 
ion, the second was a Final Reporting Conference in which the re- 
sults of the Project were disseminated to all the colleges. 

Results 




With respect to the first objective: 'dB programs were produced 
in the two summers of the project. They were field tested GLCA 
and other colleges. The programs are all relatively short, topical 
programs. Each college has received a sample of all the programs 
that were submitted to the Director's office. Eight programs are 
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currently being commercially published and probably several more 
will be published in the future. 

With respect to the second objective, five GLCA produced pro- 
grams in the fields of science, social sciences, and humanities were 
rigorously and systematically evaluated under a variety of condi- 
tions in 8 GLCA colleges, by 26 faculty members involving approxi- 
mately 1200 students. The evaluation attempted to ascertain whethe 
programs teach as well as other conventional instructional methods, 
whether they are best used before or after classroom discussion of 
the given material, whether they are used most effectively with 
lecture or with question and answer discussion and the effect.s of 
motivation on students' learning by means of programs. Characteris- 
tics of students evaluations and faculty evaluations were also 
obtained. The programs taught as well or better than traditional 
methods of teaching; the students and instructors evaluated the -pro- 
gram positively. 

With respect to the third objective a dozen commercial programs 
were evaluated. The results were largely inconclusive. 

With respect to the fourth o^Ej active the impact of preparing 
programed materials on the programers concept of teaching was in- 
.vestigated. Programing sheds light on Other aspects of teaching, 
tin addition the evaluation of the GLCA produced program described 
above also served to meet objective number 4. . j 

With respect .to the fifth objective; 42 programers were support 
ed by the Project and were intensively trained in the principles 
and procedures of programing. Six editors received further intensiv 
training to consult with the programers. An estimated 200 (about 
15% of the total GLCA faculties) were directly involved in the pro- 
ject through conferences, preparing programs, field testing programs 
evaluating them, and through contacts with the Director. All facul- 
ty members were alerted in varying degress to new media of instruct- 
ion through the initial round of faculty meetings, and two rounds 
of visits to display programs. At one point about 70 programs were 
in use in GLCA colleges, mostly in mathematics and the natural 
sciences. The results of the project were reported out at a Final 
Reporting Conference in Columbus, Ohio on April 29, 1966. Eight 
institutions and agencies other than GLCA colleges were involved 
in the Final Reporting Conference. 
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Conclusions 



1. Professors In 6LCA colleges can pre pare high quality pro- 
;grams. But preparing quality programs l^a^very time consuming 

r\vi/% n A e m ^ 



process 

2* Programs can teach successfully in all three major areas 
ithe curr iculum-*’Science j social science^ humanities* The evalua- 
!tion of the given GLCA produced programs indicated that the programn 
itaught at least as well and in some oases hatter than traditional 
jmethods of lectures, question and answer periods, tentbooks* with 
[which the programs were prepared. It made little difference whether 
the program was used before or after classroom presentation and 
'discvision of the material or whether lecture or discussion followet 
jthe use of the programed material. Students reacted favorably to 
Ithe programs they used* Faculty members evaluated the programs thej 
'Used positively* It can be concluded that good, short, single-topic 
'programs used over relatively short periods of time - two weeks to 



a month - are effective teaching 

jcollege teaching more varied and 

! 

i 



devices and 
flexible * 



offer a way of making 



3* Results of the evaluation of commercial programs were less 
Iconoluslve and less consistently positive than the evaluation of 
GLCA produced programs. The inconclusiveness of t^e results were 
idue in part to the looseness of the evaluation des|,gn* Where good 
jcommercial programs were evaluated under somewhat i^imilar condition! 
|to the evaluatioh of GLCA programs rather comparah|e results were 
obtained* The same conclusion probably holds for good commercial 



programs gts 



GLCA programs 



4, Preparing programed materials holds promise as a signifi- 
'cant procedure for analyzing and improving some aspects of teaching, 
such as, formulating operational objectives, analyzing subject 
matter, testing. The more closely involved faculty members became 
iin the process of producing programed materials the more unanimous 
and emphatic they were in their conviction that there were broad 
and useful applications of the principles and techniques of pro- 
graming to other aspects of teaching* In ^addition, faculty members 
who evaluated programed materials became more favorably impressed 
with programed materials in the process but did not necessarily gaii 
!the insights into the process of teaching that those who prepared 
programed materials . did * It can be concluded that preparing pro- 
gramed instructional materials sheds lights on otherwise unexamined 
aspects of teaching. 

1 

i Many large areas of the total teaching-learning process still 
remain to be investigated and developed. Sufficient interest, how- 
ever, has been generated in the process of teaching and the inves- 
tigation of learning to warrant a full scale attempt to understand 
them better* 
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5, The Project demonstrated the value of having an associa- 
tion of colleges prepare and evaluate programs i and the feasibility 
of a cooperative project demanding a high level of coordinationt 
and possibility of disseminating information on instructional 
matters* The Project focused the growing Interest of college teach' 
era on the process ee well as the content of teaching* It can be 
concluded that 6LCA faculties can successfully undertake projects 
demanding a high level of cooperation. 
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impiications for Further Research and Development 



The following statements seem warranted in the light of the 
experience on this project. 



1. From the value received both from the process of writing 
programed materials and from the product of the programs themselves 
it is recommended that college teachers in general and 6LCA in- 
structors in particular continue to be trained to write programs 
and to prepare programed materials~as described in this report. 



2. Programed materials as prepared on this project are ^^e 
forerunners of much more sophisticated programs of the future. It 
is recommended that means be provided for the exploration of writinj; 
and preparing programs to be used in computer assisted instruction. 



3. A valuable by-product of the project was the light shed 
upon the teaching-learning process • both in the process of writing 
programed materials and evaluating them. It is recommended that thu 
project continue to investigate the process of teaching and learn- 
ing with a greater emphasis upon learning and with the preparation 
and use of programed Instructional materials and programs for com- 
puter aosisted instruction as research tools. 
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Appen^x A-1 



SCHEDULE AND NUMBER OF CONTACTS WITH FACULTY OF EACH CAMPUS 



Dates of Visit 



Oc4:Al>er 
November 
November 
November 
November 
November 
December 
December 
December 
December 
! January 
January 
i TOTAL 



29, 1963 
5, 1963 
11, 12, 1963 
13, 14, 1963 
18, 19, 1963 
18, 20, 1963 
2, 3, 1963 
9, 10, 1963 
11, 12, 1963 
17, 18, 1963 
14, 15, 1964 
20, 21, 1964 



College 

Hope 

Ohio Wesleyan 

Denison 

Earlham 

DePauw 

Wabash 

Kalamazoo 

Albion 

Wooster 

Antioch 

Oberlin 

Kenyon 



# Faculty Contacts 

4 

12 

11 

11 

21 

6 

5 
13 

6 

11 

2 

2 

104 



The average nufliber of faculty contacts at each campus was 8*7 
persons. 



College 



Appendix A-2 

SUMMARY OF FACULTY PARTICIPATION QUESTIONNAIRES 



# of FPQs 
Returned 



)sapmk 



Number of 
Faculty 
Members 
Willing to 
Test 

Published 

Pt»ograms 



Faculty Members 
Willing to 
Test GLCA 
Programs 



Faculty Members 
Willing to 
Inspect Programs 



Albion 


34 


6 


8 


3 


jAntioch 


12 


. 4 


4 


4 


! Denison 


48 


10 


19 


17 


^De Pauw 


60 


23 


25 


16 


'Earlham 


26 


8 


13 


8 


Hope 


22 


6 


11 


6 


Kalamazoo 


8 


2 


2 


4 


Kenyon 


32 


5 


14 


15 


Oberlin 

Ohio 


18 


4 


7 


11 


Wesleyan 


46 


15 


18 


25 


Wabash 
College of 


25 


4 


6 

' • 


9 


Wooster 


36 


6 


12 

A 1 6, • ^ . 


17 

• 

* »•». « « 

... 



^ • , 4 ) 












Appendix A~3 

PROGRAMS RECEIVED BY THE PROJECT 



Decimals and Percents 


1 


Mathematics Misc 


Business Education and Economics 


6 


Sets 


Grammar and Usage > 


5 


Statistics 


Punctuation, Spelling, etc. 


3 


Trigonometry 


Language Arts 


4 


Medicine 


Modern Language 


4 


Programing 


Mathematics 


8 


Religion 


Applied Mathematics 


5 


Electricity 


Algebra 


6 


Electronics 


Geometry 


2 


Biology 


Logic 


1 


Chemistry 


* • 

1 * f 




Physics 






Psychology 






Social Studifu 

V >> 



Appendix A-4 

NUMBER OF FACULTY PARTICIPATION QUESTIONNAIRES RETURNED 



Albion. . « . . 





















4 # 



Kalamazoo 10 

Kenyon 08 

Ohio Wesleyan. 

Wabash. 26 

Wooster. 36 



3 
3 

5 
2 
1 

6 
3 
5 
1 
5 

16 
1 
7 
5 ' 
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Hope 



22 
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Appendix A-3 

PROGRAMS RECEIVED BY THE PROJECT 



Decimals and Percents 1 

Business Education and Economics 6 



Mathematics Mlsc. 
Sets 



Grammar and Usage > 

Punctuation! Spelling! etc 

Language Arts 

Modern Language 

Mathematics 

Applied Mathematics 

Algebra 

Geometry 

Logic 



Statistics 



3 
k 

4 
8 

5 

6 
2 
1 



Trigonometry 

Medicine 



Programing 

Religion 

Electricity 

Electronics 



Biology 

Chemistry 

Physics 

Psychology 



Social Studlffs 



Appendix A-4 

NUMBER OF FACULTY PARTICIPATION QUESTIONNAIRES RETURNED 



Albion. < 
Antlochc 
Denison< 



34 

9 

48 



DePauw^ 60 
Earlham. ^ 4.. 26 

Hope. **32 



Kalamazoo 10 

Kenyon 38 

Ohio Wesleyan ....47 

Wabash. 26 

Wooster. 38 



A«2 






& 



3 

3 

5 
2 
1 

6 
3 
5 
1 
S 

16 

1 

7 

5 
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PARTICIPAIITS^ IM HUMANITi: 
. ! 



February 2E-29, 1964 



WINTER WORK CONFERENCE 



Art Department 



English Dept, (cont*^i») 



William Darr 
Earlham College 



A. Ferguson 
Ohio Wesleyan 



i 







Paul Arnold 
Oberlin College 

Forbes Whiteside 
Oberlin College 

J. Stewart 
Ohio Wesleyan 

Anna Otten 
Antioch College 



Samuel Pratt 
Ohio Wesleyan 

C. Mittelstadt 
Kalamazoo College 



Dr. Harold Moon - Resource Pers 
Steven, Jordan 6 Harrison 
New York , New York . 



n 



Romance 6 Modern Language 



% 







I ■ 

- 

M’- 







Resource Person 



Dr. Dan Smith Jean Keller 
Earlham College Albion Colleg^ 










speech Department 

John Foxen 
DePauw University 

H. Sharp* 

College of Wooster 

Resource >> Prof. Donald Ecroyd 
Temple University 

English Department 

James Cook 
Albion College 

Paul Loukides 

Albion College 

• . . *1 

Kenneth Marshall 
Denison University 

Fred Bergman 
DePauw Uniirerslty 













■an 



Fred Preston 
Denison Univeisjsity 

Guy Stern 

Denison University 

LeGrand Tennis 
DePauw University 

Robert Brewster * 

Earlham College 

Charles Matlack 
Gerhard Megow Earlham College 

Hope College 

Resource - Prof. Fernand Marty 
Hollins College, Va. 

Music Department 

Carl Esehman 
Denison Univer&ity 
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Appendix A-5 (cont*d») 
Music Dept» (cont*d» ) 



i 



Floyd Peterson 
DePauw University 

Lawrence Smith 
jKalamazoo College 

jpaul Schwartz 
Kenyon Ccdlege 

Resource Person - 



General Resouce Person 

John Blyth 

Other Media 

Resource Person - Louis Forsdaie 
Columbia University [ 
New York, New York 



Dr% Robert Barnes 
Ohio State Univ. 



iPhilosophv Department 



Mort Schragrin 
{Denison Universltjf 



Harry Booth 
Ohio Wesleyan 

L. Hackstaff 
Wabash College 



Resource Person - 



Religion Department 

Lee 0. Scott 
Denison University 

Hugh Barbour 
Earlham College 

Robert Montgomery 
Ohio Wesleyiip 

K ■ 

John Blyth 

The Diebold Group, Inc. 
New York,. New York 



Appendiss A-6 

PARTICIPANTS IN NATURAL SCIENCES WINTER WORK CONFERENCE 

March 6-7 , 1964 



ijology Department 

G* Dillery 
Mbion College 

, I - 

Preston Adams 
)ePauw University 

fm. Stephenson 
larlham College 



Biology Dept. tContld.) 

Phil Crook 
Hope College* 

Francis Yow 
Kenyon College 

Donald Smith 

Ohio Wesleyan University 

i' " 

Resource Person - Dr* Lochs Vaii 
Atta, Oberlin College 
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Appendix A-6 (cont*d») 

Chemistry Department Mathematics Dept. (Cont*d, ) 



John Brown 
Denison University 

Conrad Ronneberg 
Denison University 



W. K. Moore 
Albion College 

Andrew Sterrett 
Denison University 



John MacFarland 
{DePauw University 

Owen York 
Kenyon College 

Gordon Johnson 
Kenyon College 

Peter Hawkins 
Oberlin College 



Thomas Davis 
DePauw University 

Robert Thomas 
DePauw University 

Harold Hanes 
Earlham College 

Ruth B. Smyth 
College of Wooster 



Werner Bromund 
Oberlin College 



Resource Person - Mr* Jatnee 
Evans* Teaching Materials*-' 



Richard King 

Ohio Wesleyan Uniyersity 

James Loehlin 
College of Wooster 

Resource Person - Dr. Theodor Benfey 
ACS Applied Publlcatiohs 

Geology Department 

Ansel Gooding 
Earlham College 

C. L. Bieber ‘ 

DePauw University s 

Resource Person - Dr. Dan Smith 
___ Earlham College 

Mathematics. Department 

E. E. Ingalls 
Albion College 



Physics Department 

r V 

Hugh Henry 
DePauw Unlv(|rsity 

William Achor 
Earlham College 

R. J. Stephenson 
College of Wooster 

Resource Person - Mr. Harvey 
J. Brudner, 

New York Institute of 
Technology 

Physical Education 



Betty Beese 
Albion College 



J. W. Falkenstine 
Kenyon College 



Resource Person - Mr. Stanley 
Hall* Earlham College 
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Appendix A-6 (cont*d») 



^ Library 



iMarian Mullendore 
loePauw University 



fPeter Kidder 
iKenyon College 

J. McRee Elrod 

Ohio Wesleyan University 

1 

Evan ‘Farber 
Earlham College 



Resource Person - Anne Martin 

University of Pittsburgh 

Graduate Libibary School 

■ 



General Rtsourot Parson 

Dr. James Evans 

Other Media 

John Swayze 
Columbia University 
Mew York, Mew York 



Appendix A~7 

PARTICIPANTS IN SOCIAL SCIENCES WINTER WORK CONFERENCE 



I 

( 

i 

l, 

1 

( 



j Economics Departroent 

; C . 6. Pelekoudas 
I Antioch College 






March 20**21| 1964 



Political Science Department 



L. M. Pelekoxidas 
Antioch College 



I Laurel E. Pease 

t Denison University 

! 

i) 

^ Norman Erb 
1 DePauw University 

I 

I Fred S. Silander 
I DePauw University 

I John Komives 
I Kalamazoo College 

Vant Kebker 

Ohio Wesleyan University 



Thad L. Beyle 
Denison University 

James A. Funston 
Earlham College 

Wen C. Chen 
Kalamazoo College 

Ipesource Person - Dr. John 
Ferguson 

Pennsylvania State University 
Psychology Department 'v 



Resource Person - Dr. Irwin i - Clarence Leijba 



Hernstadt, 


Antioch Col|§ige 


Northeastern University 


Boston, Massachnsetts 


Ira London 


Education Department 


Denison Uniyprsity 


Kenneth Wagoner 


Eleanor T. McLaughlin 


DePauw University 


Albion College 


Richard C. Kelly 


Clinton Green 


DePauw University 


DePauw University 


Frank S. McKenna 


Martha E. Dallman 


DePauw University 


Ohio Wesleyan University 
Harry F. Schlichting 


Rex Rector 
DePauw University 


Ohio Wesleyan University 




Joseph Wetmore 


Phillip Van Eyl 
Hope College 


Ohio Wesleyan University 
Resource Person - Dr* Daniel Smith 


Robert Brown 
Hope Cctllege 



Earlham College 
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Appendix A-7 (cont*d.) 

Psychology Dept# (cont*d4) 



Thomas Clifford 
Kenyon College 

Hubert Bonner 
j Ohio Wesleyan University 

Sam Cho 

College of Wooster 

i Resource Person - Dr* Dale Brethower 

University of Michigan 

Sociology Department 

Paul A. Thomas 
DePauw University 

Richard Knudten 
College of Wooster 

Resource Person * Professor Leslie 
Malpass. 

University of South Florida 

General Resource Person 

Mr. George Gcis, University of Michigan 

Other Media 

Anne Ro*sengr#n 
Columjbia University 
New York, New York 



History Department 

Edward Moritz, Jr. 

Kalamazoo College 

Charles Hamilton 
Kenyon College 

I • 

Resource Person -•Professor 
Leslie Malpass 
University of South Florida 



A-8 





































I 






HumanitleB 

I 

;Art 



.Speech 

I 

English 

I 

Lodern 

jLanguage 

Music 

Philosophy 
Othap Media 



Appendix A-6 

RESOURCE PERSONS AT WINTER WORK CONFERENCES 



Daniel Smithy Director of Self ~iastructional Project • 
Earlham College 

Donald Ecroyd, Professor Of Speech, Temple University 

Dr. Harold Moon, Research Associate and task leader in 
training research at the Human Resources Research Offic 

formerly manager of McGraw-Hill Programed Instruction 
Unit. 



Professor Fernand Marty, Department of Modern Language^ 
Hollins College, Virginia ‘ 

Dr. Robert A. Barnes, Assistant Professor at the Schoo 
of Music, Ohio State University 

John Blyth 

Louis Forsdale 
Johp Swayze- » 

Ann# Rosengren 



Natural Sciences 



Education 

Biology 

Ichemistry 



Daniel Smith 

Locke Van Atta, Departmeht of Biology, Oberlin College 

Dr. Theodor Benfey, Department of Chemistry, Earlham 
College 



'Mathematics Dr. James Evans 

f 

j 

'Physics Dr. Harvey J. Brudner, New York Institute of Technology 

I ^ 

Physical Education 

Dr. Stanley Hall, Department of Physical Education, 
Earlham College 




Library 



Di*. Anne Martin, Research Director, Center for Library 
and Educational Media Studies, Graduate Library School 
University of Pittsburgh ^ 
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Appendix A-8 (contM*) 



Social Sciences 



Geology 

Economics 



Dr* Daniel Smith 



Dr. Irwin Hernstadtp Assistant Professor of Economics 
College of Busines^s- Administration * Northeastern 
University* Boston 



Political Science 

J. H. Ferguson* head Department of Political Science 
Pennsylvania State University 



Psychology 

Sociology 



Dr. Dale Brethower, Center for Research on Learning 
and Teaching* University of Michigan 



Dr. Leslie Malpass* Professor and Chairman Behavioral 
Sciences Division* University cf South Florida* Tampa 
Florida 



■■ 
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Appendix A-10 



DISTRIBUTION OF AWARDED GRANTS FOR PREPARING 
PROGRAMED MATERIALS BY COLLEGES AND DISCIPLINES 



« 



a 

o 

•H 

.O 

H 

< 



Number of 
Proposals Red'd 



Art 

English 

Modern Language 
Music 

Philosophy 

Religion 

Biology 

CTiemistry 

Geology 

Mathematics 

Economics 

Education 

Political Science 

Psychology 



Sociology 
Library 
Totals by College 1 



43 

O 


a 

0 




B 

to 




0 

N 

nt 


0 




:s 


xt 




e 


•HI 


♦H 


to 


H 


e 


fd 




C 


Oa 


U 


O) 


H 


C 


e 


O 


to 


0 


«d 


< 


a 


a 




a: 




5 


8 


15 


xt 

7 


2 


4 



1 

h 



1 ih 



1 

1 



1 % 1 



k 

2h 4% 



k 

3% 1 



a 

(fi 

<D 



C 

o 

>* 

(S 

e 



a 

♦H 



m 

e 

» 



h 

e 

o 



o 

♦H 

o 



4 

m 



1 

1 



xs 

m 

xi 

m 

be 



<D. 

+x 

m 

o 

o 

» 



2 



1 * 



1\ Ik 2% 3% 1 1 



n 

H 

. m 

+i 

o 

t-* 



63 

1^ 

2 

2 % 

2 

1 

1 

2k 

1 

1 

1 

2 

1 



3% 
1 
1 
23 




ill"*”- I» addition to the above grants 
?! *«»•« awarded— one in geology and one 
in political science to two faculty renbars at Antioch College. 
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Appendix A -11 

PERSONS INSPECTING PROGRAMED MATERIALS 



Program 

Mathematics 

Modern Languages 

Physics 

Music 

Psychology 

Chemistry 

Biolrogy 

Statistics 

Logic 



llnspecting materials 
16 



9 

8 



6 

6 

4 

4 

2 



Program 
Religion 
Sociology 
Librarians 
English 
Economics 
Political Science 1 
Geology j[ 

History x 

Phys Ed (Health) 1 



#Insp0ctin|f 

2 

2 

2 

4 

1 



Appendix A -12 

SCHEDULE OF SUMMER 1964 HORKSHOFS 
‘ Workshop Schedules 

Below is the schedule for the first week's training workshop. 
Sunday^ June 14 ” 

Check into Fairlane Inn Motel by 5;30 p.m. 



Dinner - Greetings by Stanford C. Erickson. Director of th 

’'«search on Learning and Teaohine at 
the University of Michigan. -eacning at 




Evening - Demonstration and Discussions What is a program? 
Monday. June IS 

• Work through Pra cticum in Programln y ^ . 






g^sJnrprS::.::’ 
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Appendix A-12 ( cont * d . ) 

P*M« “ Finish Ppact icuin • Begin analysis of subject mattez** 

Evening - Dinner. Speaker: Charles E. Wesley, Co-ordinator 
Work Study Program, Dearborn Campus, the Univer- 
sity of Michigan. (A tour of the Fairlane Estate 
is available after dinner.) 

Tuesday, June 16 

A.M. - Analyse subject matter. Define objectives *of your 
program! Write terminal frames. Luncheon: Speaker: 
Robert R. Wilson Su]^ervisor of Correspondence Study 
and Programed instruction. Extension Service, The 
University of Michigan. Topic: Extra-Campus Instruct- 
ion and Programed Instruction. 

Wednesday, June 17 

A|M. - Discuss and write teaching frames. Dale Brethower. 
Luncheon: _Speaker - Donald B.. P. Smiths Topic:. A 
Program in Reading. 

P.M. - Continue writing teaching frames. Discussion: The 
Student as a Test Instrument. 

Evening - Homework, review and revise teaching frames. 

Thursday. June 18 

A.M. - Arrive at Ann Arbor campus to vie^ and discuss 
kinescope of testing. 



P.M. - Test programs on each other. 

t 

Evening - review/ and revise teaching frames. 

Friday, June 19 

A.M. - Discuss and plan summer work. Arrive at Ann Arbor 
campus at 12:30 for luncheon. 

P.M. - Reception at Center for Research on Learning and 
Teaching. Tour of Ann Arbor Campus. 

jBelow is the schedule of the second week's training workshop. 

Sunday, July 12 

• 

Check into Fairlane Motel by 5:30 p.m. 
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Appendix A-12 (cont*d« ) 
Dinner 



Evening - Discnssion groups discussing problems of programing 
incurred since first workshop session* ’ 



Monday, July 13 



A*M* •“ Presentation of data from test subjects to editors* 
Luncheon -» Speak**?'* David Markl’e* Topics Movijes 
and Programing. 



P*Mk - Revise and rework program and programing* 
Evening ~ Review and revise teaching frames. 
Tuesday, July 14 



A.M. - Continue revision. Luncheon Speaker: Harlan Lane* 
Topics Programing and Speech Therapy. 



P*M* •• Run paid students as subjects on program* 
Evening - Revise and rework programs. 

Wednesday, July 15 

A.M. - Final revision and field tast pla Bfl ing. 
P.M. - Discuss program management and evaluation. 
Evening - Revise and rework program. 

Thursday, July 16 



A.M. “ Speaker: D. Bern. Topic: Programing for Mentral Retar4 
dates. R. Richards. Topic: Listening Program. 



P.M. - Free time for tours, etc 



Friday, July 17 



A.M. “ Speaker: Dale . Brethower , Georf;e Gels, 
in Programed Instruction. 



Topic: Research 



P.M. “ Luncheon and adjournment. 
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Appendix A-13* 

REPORT OF CHEMISTS FROM THE WINTER WORK CONFERENCES* 
Distributed to Department Chairman 

Report on Chemistry Discipline Group 

Present! John Brown, Denison; Conrad Ronneb.erg, Denison; John 
McFarland, DePauw; Owen York, Kenyon; Gordon Johnson, 
Kenyon; Peter Hawkins, Oberlin; Werner Bromund, Oberlin 
Richard King, Ohio Wesleyan; James L o e h 1 in, Col lege of — 
Wooster; Ted Benfey, Earlham, Resource Person 

Available Programed Materials in Chemistry 

Most published or mimeographed programs were on view at the 

1 prepare a li^ of all known chemistry programs. 

The list will be sent to all participants in the chemistry group. 

The large majority of the programs are either remedial or 
adjunct programs. Very little work has been done so far in trans- 
I standard lecture material to programed form. Almost all ‘ 

the chemistry programing is on the freshman level. 

Testing Present Material 

1. King plans to try out Halter Hunter's program on Symbols and 

Circuitry, Earlham, in his inctrumental analysis course, this 
spring. * 

2. McFarland plans to try out Benfey 's Aliphatic Nomenclature 
programs this spring. 

3. Next fall, Johnson and York are planning to test parts of 

Barrow s program; King will try out parts of either Barrow 
or Jay Young. 

Preparation of Programs . 

1. Brown proposes full-time programing this summer on! 

a) Phase Rule And its applications 

b) One or more physico-chemical derivations and their applies 
tions such as Clausius-Clapeyron equation, Boltzmann dis- 
tribution law, Debye-Huckel equation. 



I 
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4 . 



fof ceriain^? 2 s?rumen?o^''^ ^onsidering prograning instructions j 

canic”eh»ml of programing preliminary material in or- 
that l«ctures^Ln°very^quicLy^foLs^ Cram and Hammond text s 

and a skeleton set of reactions interrelating function“rgroupJ. 



>ome Topics Discussed 
Types of Programs: Linear 



Washbacks , Optional Fram 



vs 



es, etc. 



Intrinsic} Linear, with Loops; 



Purpose s of Programs in Chemistry Teach i nf. 



A ) P 6 mnii t soiBi 6 irh ins of 9 socr'ai*?^-* ^ 1 ^ • n 

b) Reduce the large diversi^rof u classes, 

beginning the fourse! chemical knowledge of studen- 

a) OrI!I ItTflt Class. 

in class. udents, extra samples beyond those discusi 

oueatJon^’rf^"®? value of programing for this purpose was 

^ o’r?sr::?e“^r:af of%rLen%:rion 

^ matica? p^epa^iLnr*® “'’emistry, e.g. mathe- 

m«eriarw.°ich^ifnof ( logically coherent 

material w.nich is not learned effectively through lectures 



fee 



S ome Areas _Hhere Pro g rams Are likely to he Useful 
For Freshman: Al€rAKvna_ * 1 ) ^ . -*- A 



Freshmen! *j 8 ebra, linear equations, direct and inverse proper-! 

louarltSL f^fPhical expressions, exponents, ' 

litlh ?h:rmXa":?cr^“^^"® 

ample^orJrivIousJriearLfThior^U^ **- 
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Appendix A-13 (cont»d.) 

Honenclatuf.! Organib - Stereochemistry, simple functional grouo 

chemistry 

Analytical - Electronics for chemists: instructions 
for use on instruments. 

Physical Chemistry - Mathematical preparation for 
physical chemistry, physicochemical 
derivations • 

rollo«-up! Mo formal organitstion was set up.- Programers and 

program testers will plan to keep in touch with othei 
in the group involved in similar projects. 

Materials or information for the whole group or for 
all chemistry faculty in 6LCA can be sent to Dr. 
Benfey or Dr. DeHaan. 

We^ht programers and testers might 

jmeet at MACLAC and Ohio Chemistry Teachers gatherings. 



Lr,^ royalty questions would need to be settled 

pnd spelled out so as to prevent confusion and misunderstanding 
among summer programers. » tauamg 

I 

i 0* Theodor Benfey , Consultant' 

j 1155 - 16th Street, M. W. 

I Washington, D. C. 20036 

j 

1 In addition 13 programs in chemistry were reviewed ani the 
reviews were distributed to all the dhemistry department chairmen. 
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Appendix B-1 

INSTRUCTOR PROGRAM EVALUATION FORM 



Date 



Course 



2. Institution 



I 3* Name of Program 



t 



1 Prior Experience with Programed Material 

u. 



I 



! 5 . 



Have you ever used programed 
material before this time? 4. 

If the answer is yes to question 
4, answer S and 6. 

In what cor^^ses did you use 
programed material? 5. 



Yes 



No 



6 . 



As a reference work 



f .. 



As a primary teaching 

device in place of lecturjii 

% 

For remedial <«ork when thh 
student "falls behind 



for : enrichment for stu- 
dents who want more 

t 

To bring students up to 
starting level of the 
course 

To teach material that 
the instructor does not 
want to cover in class 



other (describe it) 
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Appendix B-1 (cont*d«) 



^ • Describe your attitude toward 
programed instruction prior 
to evaluation of the material 
for 6LCA • Comment if you 
wish: 



• E nthusiastic 

F avorably disposed 
U ncertain 

U nfavorably disposed 
S trongly disapprove 



General Reaction 

Will you please describe in your own words your reaction to and 
evaluation of the programed material you used. More specific 
questions will be asked later in this questionnaire t and' you may 
wish to return to this question after you have responded to theme 



Please use the back of this sheet if yon wish to make more com- 
ments than the above space allows. 



specific Reaction (Add any comments you wish to your answers) 
B. Does the program cover essentially the 9. yes Mo 

same material that you desire to cover — 

in part or the whole of your course? Comment: 



LO. Are there any errors in the content? 

LI. If you answered yes to question 10, 
are they major or minor errors? 



10. 


Yes 


Mo 


11. 


Major ^ 


Minor 




^Both 





Give examples of frame number and page 



B-2 
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Appendix B-1 (cont*d.) 



1 



12* Are there any technical program 
errorSf e*g*( poor directions* 
points of confusion* etc? •12* 

/< Comment: 

Give examples by frame number and page: 



Yes 



No 



13. In your opinion does the program actually teach 13, 
what it claims to teach to the student for 
whom it is presumably designed? 

If your answer is "No" or "Partly" to Question 13 
above* what claims are exaggerated? 



Yes 

Mo 

Partly 



14, Are the responses that the student is 14, 
required to make token or real? (That 
IS* do the frames merely required 
mechanical filling in or do they re- 
quire thought and/or problem solving)? 



15, Does it happen that the same response 
it required again and again in a 
series of frames? 

16, Evaluate the pace of the program 



15. 



16. 



1 17, Evauate "step size" or amount of in* 
I formation presented in each frame. 



17. 




;16. Does the student get a sense of 
direction where he is going from 
the program? 



B-^3 



18. 



__^Mostly token 
Responses 
^Mostly real 
Responses 
^Spotty* some of 
*’both 

Yes 

‘No 

^Occasionally 

JToo fast 
JToo slow 
Uneven 
About Right 



^Too big 
JToo small 
JUneven 
jAbout right 

Yes 

yo 

At times 



ERIC 
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AppendiK B-1 (cont*d.) 

ii J N il9. Can the student easily review the 

V ^ > programed material? 



t 

HliV 



! 



}20. Are criterion (terminal) frames 
I identifiable? (Criterion frames 



H '' 



,v 



o 

ERIC 



not present new information 



l> 

\ 1 21. Does the importance and amount of 

I content covered justify the amount 



of time required of the student? 

22. Does the program save the instructor 
I tine? 



23. Is the program justified in terms 
of its cost? 



24. Is the following information about 

the program available? 

a. Was the program adequately tested 
prior to publication? 

b. If so» does it describe the 
characteristics of the students 
on whom it was tested? 

c. Are the results of the testing 
given? 

d. Are pre and post-test available? 

25. Do you know of better or similar 

programs or comparable material? 

If yes, what? 



19. 



20 



21 . 



22 . 



23. 



Yes 

‘No 

jAt times 

Yes 

“No 

“At times 

Yes 

“No 

__Uncertain 

Yes 

“No 

“uncertain 

Yes 

“No 

“uncertain 



24. yes uncertain 



no 



b. 



d. 



26. Does the teacher's manual or the in- 
troduction to the program state ex- 
plicitly the objectives of the- 
program? Comment 



25. 



26. 



Yes 

‘No 

Yes 



No 



27. Does the program state explicitly 
tha prerequisites or "entering be- 
haviors" for the student for whom 
it Is designed? Comment 



27 



Yes 



No 
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Appendix B-1 (cont'd.) 

28. Would you prefer to have students 28. 
use a regular textbook or a pro- 
gramed version of the material? 

Comment if you wish 



29. As a method of presenting materialt29. 
would you prefer to lecture or the 
use of programed material? 




30. Estimate total number of hours you 

spent on testing these programed 
materials, including conferences 
with director, etc._^ _hrs. 

31. In summary, what is your overall 31. 
reaction at this time to using 
programed material in this course? 

Comment if you wish 



Prefer programed material? 

Indifferent 
Prefer lecture 

i 

) 

I 

f 

Prefer programed ma^rlal 

, , I,... 

Indifferent 

Prefer lecture { 



V ery positive, would 
like to use similar ma- 
terial again 

M ore positive than 
negative reaction to it 
Indifferent to usin ; 
it 

M ore negative than 
Positive reaction to it 
V ery poor, would 
object to using it agaiii 



\ 



f 
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Appendix B-2 



STUDENT REACTION QUESTIONNAIRE 



Fall 19o4 



As you may know, you have been participating in a research project 
designed to compare the use of programed instructional methods with 
usual methods of presenting materials in a college course. We 
would like to compare your reactions to the methods used in your 
course to the reactions of other students who have used programed 
materials. Will you kindly answer the questions about the methods 
and materials below? This questionnaire is to be used for research 
purposes only and will not affect your grade. 



Name 



Date 



Course 



2. Name of program 



3. Your major or intended major 
Your career choice 



Prior Experience, if any, with Programed Material 



4. Have you ever used programed 
material? 






Yes 



No 



If you answered yes to question 4, 
answer 5 and 6. Otherwise go on 
to the next page. 



5. Where have you used programed 
material? 



5. 



_High School 

_In how many courses? 



^College 
_In how many courses? 
Home (describe it) 



GT^ow was the programed material 
used? 



6 . 



As a reference work 



_As a primary teaching 



device in placeyp:^ 
the textbooks 



jAs a primary teaching; 
device in place of 
lectures 



jKb a supplement to 
textbooks and lectures 






1 
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Appendix B-2 (cont'd*) 
6. (cont’d.) 



For reaiedial work 
when the student falld 
behind 



_^For enrichment for stu 
dents who want more 



Jo bring students up 
to starting level of 
the course 



Jo teach material thai 
the instructor does 
not want to cover in 
class 

Other (describe it) 



E^valuation of Educational Methods Used in this Course 




,i’ 





7, Check the term(s) on the right 
that most adequately describe 
the major educational methods 
used in this courser If more 
than one method was usedy write 
a numeral 1 in front of the one 
that received major emphasis. 



7 • l ecture 
t extbook 

d iscussion groups 

independent study 

laboratory 

field work 

other (describe it) 



In answering the following questions, use what you have checked in 
Question 7 as your point of reference. For example, if you checked 
"textbook” as a major educational method in 7, let the rest of your 
answers apply to tekt books. 
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Appendix B-2 (cont*d.) 



< 8 . 



'9. 



ilO 



er|c 



What is your overall reaction at 
this time to the educational 
method(s) used in this course? 
Comment if ycu wish 



As a method of being exposed to 
knowledge and information* how 
would you rate the educational 
methods used in this course? 
Comment if you wish 



9. 



Considering what you learned from 
the methods in this course* was 
it worth the time you spent on it? 
Comment if you wif»ki 



10 . 



fll. Did you have a sense of direction. 11. 
of knowing where you were going* a 
feeling of getting somewhere in the 
course? Comment if you wish 






Very positive learning 
"experience* would like 
to use similar again 
_More positive than 
negative reaction to it 
^indifferent to using ijt 
_Mp.re negative than pos^ 
itive reaction to it 
^Very poor learning ex- 
perience* would* object 
to using it again 

Very effective 



Usually effective 



Mixed 



■Usually ineffective 



Vei!’y ineffective 



Yes 



Uncertain 



No 



Yes 



Most of the time 



Sometimes 



Usually not 



No 
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Appendix B-2 (cont'd.) 



12. Could you easily review the 
course materials? 



12 . 




Yes 

Uncertain 



No 



Charaterize^tha metli6d(s)u8e‘d in 13 
this course as you experienced 
it. Check those statements 
that apply. 



^Speeded my learning 
Impersonal 



^Gave me a feeling of 



making progress 
Mechanical 



Challenging 



Too hard 



^Other (describe in foui> 
own words) i 



[ 



4. 



Characterize further the met*-* 
hod(s)'u8ed in this course as 
you experienced it. Check 
those statements that apply. 



14. 



_too easy to skim 
JJot worth the time I 
"put into it 
_Made me think 
^Too easy 

_Dld not hold my atten- 
’tion 

_In spite of any disad- 
vantages, I learned 



much 

Other (describe it) 










\ 







▼ 






X 






o 

ERIC 










Apptndix B-2 (eont*d.) 

15* As you 80 S it now* whot is your 
prefsrrsd way of being exposed to 
knowledge and information in your 
atudiaa» Place a numeral 

1 in l^roThr of your if irot“* choice,, 

2 in front of second choice • etc. 



15. ^lecture 

[textbooks 
jdiscussion groups 
.independent study, 
supervised 
.laboratory 

.Writing research paper& 
.field work 

^informal bull sessions 
.unsupervised, fz^ee 
reading 

Other (describe it) 



16. How has the use of the methods 
employed in this course affect- 
ed your attitude toward the 
embject matter of this course? 
Comment if you wish 



16. 



J^avorably 



.Mo appreciable effect 
• Onfavorablv 



17. Please w<*ite any other specific or general comments you wish 
on the back 
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Appendix B-3 
6LCA FIELD TESTERS, 1964 



Programer 

1. Prof. Preston Adams 
Dept, of Botany 
DePauw University 



I 

Field Testers 



2. Prof. R. Brewster Dr. E.F, Gearhart 

Dept, of Languages Dept, of Languages 

Earlham College Hope College 



3. Prof. James Cook 
Dept, of English 
Albion College 

4. Prof. T. A. Davis 
Dept, of Math 
DePauw University 



Dr. Robert H. Ross 
Dept, of English 
Ohio Wesleyan U. 

Dr. W. Keith Moore 
Dept, of Math 
Albion College 



'i. Prof. Ansel Gooding 
Dept, of Geology 
Earlham College 

6. Prof. Peter Hawkins 
Dept, of Geology 
Earlham College 

7. Prof. Vant Kebker 
Dept, of Economics 
Ohio Wesleyan U. 



Prof. C.L. Bieber 
Dept, of Geology 
DePauw University 

Dr. Theodore Benfey 
Dept, of Chemistry 
Earlham College 

Prof. F. Silander 
Dept, of Economics 
DePauw University 



Dr. Walter ^»^aring 
Dept, of English 
Kalamazoo College 

Dr. Andrew Sterret 
Dept, of Math 
Denison University 

Dr. Harbld Hanes 
Dept, of Math 
Earlham College 

Prof. G.H. Crowl 
Dept, of Geology 
Ohio Wesleyan U. 









8. Prof. Richard Kiiudten Prof. Don Clelland Prof. Thomas Lawrence 

Newbe^y College Dept, of Sociology Dept, of Sociology 

Newberry, S* C. Hope College Newberry College 

9. Prof. Clarence Leuba Dr. Frank McKenna P*»of. F. Grohsmeyer 

Dept, of Psychology Dept, of Psychology *’PepTv-^<l>^ Psytir ol e gy ' 
Antioch College DePauw University Earlham College 

10. Prof. W.M.Lotkowski — — 

Dept, of Earth Sciences 
Antioch College 









B-ll 



























Appendix B-3 (cont'd.) 

!!• Prof. Kenneth Marshall 
Dept, of English 
Denison University 

12. Prof. Frank McKenna 
Dept, of Psychology 
DePauw University 

13. Prof. R. Montgomery 
Dept, of Religion 
Earlham College 



14. Prof. L. Pelekoudas 
Political Science 
Antioch College 

15. Prof. Morton Schagrin 
Physical Science Dept. 
Denison University 

16. Prof. P. Schwartz 
Dept, of Music 
Kenyon College 

17. Prof. F. Silandsr 
Dept, of Economics 
DePauw University 

18. Prof. Lawrence Smith 
Prof. Ray Hammar 
Prof. Harry Ray 
Dept of Music 
Kalamazoo College 

19. Prof. H. Stephenson 
Dept, of Biology 
Earlham College 

20. Dr. E.P. Van Eyl 
Dept, of Psychology 
Hope College 



Prof . F . Grohsmeyer 
Dept. of Psychology' 
Earlham College 

Prof. Hugh Barbour 
Dept, of Religion 
Earlham College 



Prof'. H. Bi.hrlck 
Dept, of Psychology 
Ohio Wesleyan Unlv; 

Prof. Lee Scott 
Dept, of Philosophy 
Denison University 

Dr. M. Vulgamore 
Dept, of Religion 
Ohio Wesleyan Univ. 



I 



Prof. L. Hackstaff 
Philosophy Dept. 

Wabash College 

Prof. Karl Eschman ?TQf • L. Holvik 
Dept, of Music Dept, cf Music 

Denison University i|rlham College 

Prof. R. Montgomery Prof. Morman Erb 
Dept, of Economics Dept, of Economics 
Denison University DePauw University 

Prof. David Strickler 
Dept, of Music 
Albion College 



Dr. Donald Smith 
Dept, of Zoology 
Ohio Wesleyan Univ. 

Prof. D. Whitted 
Dept, of Education 
Ohio Wesleyan 



Prof. Francis Yow 
Dept, of Biology 
Kenyon College 

Prof. James McDowell 
Dept, of Physcholoy 
Earlham College 
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Appendix B->3 (contM.) 

1 21. Prof. Joseph Wetmore Prof. Miltop Kraft Prof. Donald Orlosky 
I Dept of Education Dept, of Education Dept, of Education 

Ohio Wesleyan Earlham College DePauw University 



22. Prof. F. Whiteside 
Dept, of Art 
Oberlin College 






.‘ri 







Sunday, June 13 



Appendix B-4 

AGENDA OF FIRST TRAINING WORKSHOP 
Summer 1965 



7:30'P.M. Planning Meeting; all editors 






Monday, June 


14 




9:30 


* 


11:00 


A.M. 


11:00 


- 


12:30 


P.M. 


12:30 


•• 


1:30 


P.M. 


1:30 


- 


2:00 


P.M. 


2:00 


• 


4:00 


P.M. 


4:30 




5:15 


P.M. 


■5:15 


mm 


7:30 


P.M. 


7:30 
6 : 15 




8:15 


P e M e 


Tuesday,' June 15 


8:30 


- 


12:00rA/.M. 


12:00 




1:00 


P.M. 


1 : 00 




2:00 

/ 


P.M. 


2: 00 


- 


5:00 


P.M. 


5 : 00 




7:00 


P.M. 


7:00 


- 


8 : 00 


P.M. 


8:00 








Wednesday 


- - ■ 

, June 16 


■—•8:30 


• 


12:00'’ 


■'A.M. 


12 : 00 


mm 


JL j Q Q 


P.M. 


1:00 


- 


2:00 


P.M. 


2: 00 


- 


5:00 


P.M. 


5:00 


- 


7:00 


P.M. 


7:00 








Thursday, 


June 17 




8 : 30 


- 


11 : 00 


A.M. 



Review of working outlines of first contiifi- 
gent of programers. Discuss formulation of 
behavioral objectives. 

Review outlines*^*! second contingent of pro- 
graimers. Begin formulation of behavioral 
objectives. 

Lunch 

Optional group discussion of how to formulat 
behavioral objectives. 

Work on outlines, behavioral objectives. 

Dan Smith, "The Process of Programing" 
(correct title?) 

Break, recreation, dinner 

Movie, "Program Construct ion" (correct title? 
Consultation with editors, formulation of 
objectives 

Construction of criterion questions, consul- 
tation with editors 
Lunch 

Optional group discussion of criterion be- 
haviors and criterion test frames. 

Write sample criterion tj| 9 ts for one or two| 
objectives, consultation editors. 

Recreation break, dinner* 

Optional group discussion writing and se- 
quencing teaching frames. 

Begin writing teaching freuines, consultation 
with editors 

Writing teaching frames, consultation with 

editors 

Lunch 

Optional discussion by editors on process of 
using test subjects 

Test out program on colleagues, consultation 
with editors 

Sign out, make arrangements for editing - 
session between workshops. 

Consultation with editors 

"Spill-over-editing" session (optional). 



) 
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Appendix B-5 

SAMPLE FORM OF EVALUATION DESIGN 

1* Evaluator: Frank Yow, Biology l)epartinent i, Kenyon College 
2* Program: Biochemistry for Biologists by William Stephenson 
3. Course in which program is to be tested: Two sections of Intro- 
ductory Zoology* Mo# of students 60 - one large class* 

Inclusi/e dates of experiment: September 20 - October 11 (reten- 
tion test October 25) 

Design #1 Group A versus Group B 6 c 

Evaluation Question: Do programs teach as effectively as text- 
books and lectures covering the same material? 

Details of Design 

a# Designate one section as Group A and one section as Group 
b* Day to day procedures are as follows: 

Period 1 Period 2 



5* 

6 . 

7. 



B 



Period 3 



Gr«A 



1st week Gr*B 



1, Give Pre-test 1. 
2* Assign Program 
to be completed 
by period 2 of 
2nd week 

1* Give Pretest 1. 

n 

2* Lecture §1 and 
textbook 



no class #1 1* no class 



Lecture #2 
and text 
assignment 



1* Lecture and 
text assign- 
ment 



2nd week 



3rd week 



i 



pi 



Hi 

f V 

E 

'^-1 

& 



6r *A 


1* no class 


1* 


Attitude 1* 


Get 5 written 1 


1 








Quest ion’-i*i * 


questions from 


1 








naire 


each student 


i.. 






2. 


Post test 2. 


Discuss materic 


.1 1 




1 




n 


-• 


■3 

m 


Gr.B 


1* Lecture i^4 


1. 


Attitude 1* 


Get 5 written 


1 




and text 




Question- 


questions from 


T-i 

ij 




assignment 




naire 


each student 








2* 


Post Test #2 2 * 


Discuss materic 


1 . 1 


Gr • A 


Discuss 


1* 


Give Retention 








material 




test #1 










2. 


Give Attitude 












Questionnaire 




i 


Gr.B 


Discuss 


1* 


Give Retention 








material 




Test §1 




y 






2* 


Give Attitude 






* 






Questionnaire 




ii 

if 

1 ^ 
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Appendix B"5 (contM.) 

0. Specific responaibilities of evaluator 
a 



b. 
o • 



Tape record the discussions in class (including the names of 
participants. ) 

Submit a written copy of all lectures to the project coordinatlor, 
Give a subjective evaluation of the quality of discussion in 
class and of the written questions of the students. 

9. The evaluator will receive a stipend of $500. plus money for 
materials and clerical help. It is anticipated that about 10 
days of the instructor’s time will be required for the evaluation 
including conference time. The stipend will be paid when .all 
data have been collected and forwarded to the research cooi^diria- 
tbr for processing by computer. 

Appendix B-6 

GENERAL RESPONSIBILITIES OF ALL PROGRAM EVALUATORS 

iThe following checklist indicates the general responsibilities that 
each evaluator should carry out in conducting the program evaluation 
*1 1. Collect student personnel data and see that it is recorded by 

[ groups on the Personnel Data Roster Forms. 

Administer and score all tests required in the research design 
Administer student questionnaires according to research design 
and insure that they are completely filled out. 

Complete instructor quiestionnaire . 

Send completed Personnel Data Roster forms, student question- 
naires and the instructor questionnaire, in one package, to 
the research coordinator at the following address; 

Professor Donald Deane 



2 . 

3 . 

4. 

5. 



6 . 



Submit bill for cojt of 
project to the research 



8 



272 Kerr Drive 
Hilliard, Ohio 43026 
clerical help and materials for the 
coordinator. (Maximum of $150 
allowed for clerical help on each project.) 

Follow evaluation design explicitly. If you have questions 
call the research coordinator collect, Phone (614) 876-5890. 
Attend Training' Conference at Holiday Inn, Columbus, Ohio, 
Airport, Thursday, 9:00 A.M.(EDT) to Friday noon, September 9 
and 10. Notify Don Beane in Hilliard if you want a single or 
double room reservation for September 9. Reservatiofis should 
be made at least one week before conference. 






'V-- 
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Appendix B-7 



AGENDA OF PROGRAM EVALUATION TRAINING CONFERENCE 
Holiday Inn of Columbus Airport 
September 9 and 10, 1965 



10:30 

11:30 



n 



I Thursdavt September 9, 1965 
' 9:30 A*M* Registration and Coffee Hour 

Progress Report on GLCA Programed Instruction Projec 
(Film on Programing) -- Robert DeHaan 
Procedures for Fall Testing of 1964 GLCA programs 
--Donald Beane 
Lunch 



II 



12:30 

1:30 

-4:30 



P.M.*. 

II 



6:00 

6:30 

7:30 



II 

II 

II 



Group discussions of specific programs to be tested 
with the program authors (includes coffee break) 
Distribution of programs y tests, questionnaires, etc 
Dinner 

Beane available for discussion of individual eval- 
uation designed 



Friday, September 10, 1965 



8:00 

9:00 



A.N« 

II 



10 : 00 
10:30 



II 

ii 



l^reakfast 

Logic of Research on Evaluation of Programing 
Morton Schagrin 
Coffee Break 

Computer Treatment of data and interpretation -- 
William Jensen 
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PARTICIPANTS AT THE EVALUATOR »S CONFERENCE 



Evaluator 



College 



Dates of Evaluation 



■1 

1, 


P. 


Church 


Kenyon College 


Nov* 16— Dec* 16, 


1965 


iT. 


Burkett 


Ohio Wesleyan University 


Jan* 3-19, 1966 




1 


T. 


Boyle 


Albion College 


Nov* 22-Dec* 16, 


1965 




Cook 


Albion College 








D. 


Anderson 


Ohio Wesleyan University 


Sept* 20-0ct* 8, 


1965 


i 

1 








and Jan* 4-18, 1966 


B. 


Angel 


Ohio Wesleyan University 


Oct* 16-28, 1965 


and 


1 








Jan* 21-Feb* 11, 


1966 


! 


P. 


Van Eyl 


Hope College 


Sept* 27-Oct. 8, 


1965 




R. 


DeHaan 


Hope Collage 


Oct* 5-19, 1965 




j 


J . 


Tovo 


College of Wooeter 


Sept* 25-Qct* 2, 


1965 


ft' 


M. 


Schagrin 


Denison Uni'^arsity 


Sept* 17-Oct* 4, 


1965 
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, Evaluator 


College 


Dates of Evaluation 


t 

t 

L. 


Wilcox( 2oval) 


Earlham College 


* 1 

Oct. 1-Nov. 5, 1965 , 


F. 


Yow 


Kenyon College 


Sept. 20-0ct. 25, 1965 


W. 


Patton 


Ohio Wesleyan University 


Sept. 17-Oct. 2, 1965 I 


,v. 


Wilson 


Denison University 


Oct. 18-Nov. 17, 1965 


»>W. 

1 


Stephenson(2) 


Earlham College 


Oct. 1-Nov. 5, 1965 


!d. 


Stegner 


Illinois Wesleyan Unlv. ' 


Oc^. . 4-20, 1965 


jM. 


Vulgamore 


Ohio Wesleyan University 


Nov. 1-15, 1965. 


*L. 


Scott 


Denison University 


Jan. 3-21, 1966 


‘J. 


Stone 


Illinois Wesleyan Unlv. 


Jan. 5-7 ,“1966 


»>R. 

j| 


Montgomery 


Ohio Wesleyan University 




S. 


McNaghten 


Denison University 


Sept. 20-Qct. 4, 1965 


|b. 


Westbrook 


Denison University 


Oct. 1-9, 1965 


|d. 


Beane 


College of Wooster 




Wm . 


Jenoon 


Kalamazoo College 




R. 

1 


DeHaan 


6LCA Director 




jlhe 


i following were absent with excuse: 




,w. 


Judd (Eng . ) 


Ohio Wesleyan University 


Sept, 22-Oct. 13, 1965 


1C. 


Weis (Eng . ) 


Ohio Wesleyan University 


Sept, 22-Oct. 13, 1965 


R. 


Mizer (Eng.) 


DePauw University 


Sep^. 13-27, 1965 


jR . 


Johnson (Eng.) 


DePauw University 


Sept:^ 13-27, 1965 


T. 


Wirt (Govt.) 


Denison University 


Sept, 20-0ct. 4, 1965 


R. 


Morey 


Denison University 


Sept; 20-0ct. 4, 1965 


*L. 

1 


Pelekoudas 


Antioch College 


> 



A 



Authors of the programs t© be avadriiated who were on hand to ©onfer 
with the evaluators about the programs i pre-end post-tests and 
evaluation designs. 
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MEMORANDUM 

TO: Evaluators of 1964 6LCA Programs 



FROM: Donald 6. BeanOf 
OoortliTrair'OT" 



The conference held at Columbus Airport September 9 and 10 
clarified some points and resulted in a few changes which we want 
everyone to observe* Listed below are the items which affect almosjt 
every design: 

!• Under deisgn #1 the retention test (form #1 used a s’econd 
time) should be administered at least one week but not 
more than two weeks after the last discussion period. 

2. The Attitude Questionnaire (Teaching-Method Evaluation 

Questionnaire) will be administered only once in all cases 
This should be done as near the end of the experiment as 
possible* But do not wait until the retention test to 
give the attitude questionnaire ^ the retention test Is 
^ scheduled) at least a week after the end of the experiment 
— (TTB'"itnQesign #1). 



3. If the attitude Questionnaire is given diiring the sam© 
period as a test, the Attitude Questionnaire should be 
administered first so that the students' reactions to the 
test will not influence the responses on the questionnaire 

4* All tests supplied by the project and usfd in more than 
one design (pre-tests, post-tests, retention tests, final 
tests) should be administered with a 40 minute time limit* 

We cannot compare results on a test if one evaluator gives 
his students 40 minutes, another 45 minutes and perhaps 
another, 60 minutes* The 40 minute time limit may be shor-: 
ened for the pre-test since students will not be able to 
answer many of the questions and consequently will not 
need as much time. 

5* Evaluators using Design §1 are to ask students for written 
questions which form the basis for subsequent class dis- 
cussion* These written questions should be forwarded to 
the coordinator along with the tapes of the class discussi(^n 
The evaluators are to give a subjective evaluation of the 
quality of the questions submitted by students in the two 
or three groups being compared* The class discussion under 
design #1 is also to be evaluated for differences in qual- 
ity between the groups being compared* This evaluation is 
a part of design #1 only* 



7. 
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6. Following the administration of the post-test in ea^ eval- 
i nation study the programed materials are to be collected so 

i that students cannot refer to them during the subsequent 

I class discussions. The programs may be returned to the 

students when the experiment is completed. Obviously ir 
the post-test completes the experiment, the programs need 
not be collected. 

Each evaluator will receive a copy of the IBM print off 
giving the statistical analysis of the data for his ‘study. 
Each programer will receive a copy of the statistical anal 
ysis of the studies using his program. 

8. If the design calls for a section to be divided in^;^ two 
groups, we have asked that the selection be done randomly. 

If this results in obvious differences in the composition 
of the two groups which could account for significant 
ferencesiin achievement on the tests, please explain the 
•• -situation somewhere on the instructor questionnaire. But 
do not alter the composition of groups once selected, in 
an effort to match them on some variable. Differences may 
also be evident between sections of the same course due to 
class scheduling, college activities or some other variable 
unknown to us. Your making notes of these observations will 
help U8 account for differences in group performances which 
are not due to the research design being used. Administer 
the pre-test as orginally planned. 

) 

I 
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LIST or FIHAL REPORTING '^COKFERENCE PARTICIPANTS 
GLCA College Participants GLCA College Particip a n ts (cenfd 



.) 



Albion 



Renato Gonzales 
James Cook 
William Gilbert 
Jean Keller 
Robert Lisenski 
William Cowell 



> nt ioch 

Richard Meisler 



Denison 

George Gilbert 
Samuel Shaff 
Ira London 
William Westbrook 
Thomas Gallant 
Morton Schagrin 
John Morris 
Irvin Wolf 



John White 
Don Myatt 
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GLCA College ParticipantsCCont *d . ) 



DePauw 



Thomas Davis 
Garrett Boone 
Donald Orlasky 
Ray Nlzer 
William Hanlon 
Richard Montgomery 
Frank McKenna 



1 Earlham 



Hope 



Robert Brewster 
William Stephenson 
Alfred Henderson 
Daniel Smith 



Philip Crook 
Frank Sherburne 
Elliot Tania 
Douglas Meekers 
Ralph Perry 
Philip Van Eyl 
Leslie Beach 



Kalamazoo 



Walter Waiting 
John Moore 

Kenyon 

Peter Kidder 
Paul Schwartz 

l Oberlin 

Forbes Whiteside 

Ohio Wesleyan 

Joseph Wetmore 
WendeX Patton 
Robert Montgomery 
Daniel Andersoh 
Melvin Vulgamdre 
J. HoRee Elrod 



] 



LCA College Participants 

(Cent * d # ) 

College of Wooster 
James Hawley 
Harry Sharp 
Dean Garber Drushal 
John Baker 
Jerry Tovo 
Donald Wise 
Roy Haynes 
Bud Russell 
Sam Cho 

Task Force Members 



Albion 

Antioch 

Denison 

DePauw 

Earlham 

Hope 

Kalamazoo 

Kenyon 
Oberlin 
0. Wesleyan 

Wooster 



James Cook 
Jean Keller 

Richard Meisler 
John White 

George Gilbert 
Samuel Shaff 

Thomas Davis 
Garrett Boone 



Daniel Smith 
William Stephenson 

Elliot Tanis 
Douglas Meekers 

Ws>lter Waring 
Jo. n Moore 

Paul Schwartz 

Forbes Whiteside 

Joseph Wetmore 
Robert Montgomery 

Don Wise 
Sam Cho 
Donald Beane 
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LIST OF VISITORS 

Professor Algo Henderson 

Center for the Study of Higher Education 
University of Michigan 



Dr. William Demlnoff 

Commission on Institutional Cooperation 



jProfessor Forbes Robertson 
Prlncipia College 
Elsah» Illinois 

I Mr. James Brink 

Center for Research on Learning and Teaching 
University of Michigan 

Or. Donald Liggett 

Associated Colleges of- the Jlidwest 

Mr. Kenneth Templeton 
Lilly Foundation 



Professors John VandenBerg, Dennis Hoekstra, Marion Snapper 
Calvin College 

Professor Guy Stern 

University of Cincinnati 



Robert F. DeHaan» Conference Director 
Mrs. Robert DeHaan» Conference Secretary 
Prof. C,larence Leuba» Wright College Campus 
Prof. So'pa'vld' Beane » College of Wooster 
“~TjrT^^don Johnson • President 'of GLCA 




CSS 




B-22 



ERIC 



Appendix B-11 



AGENDA FOR FINAL REPORTING CONFERENCE 

Place: Holiday Inn 

Columbus, Ohio 
Date: April 29, 1966 

Objective of Conference: Report out the results of Project, 

develop plans for further study and 
improvement of instruction 



Friday, April 29 


8:30 


a.iii. 


9:00 


If 


9:15 


U 


10:15 


19 


10:30 


If 


11:15 


If 


12: 00 


p • in • 


1:00 


u 


1:30 


• 1 

19 


3:15 


If 1 


3:30 


U 


5:00 


M 


6: 00 


M 


8:00 


M 



Registration and program display 

Welcome and opening remarks, Robert DeHaan, 

Project Director 

"Evaluation of GLCA Programs,” Donald Beane, 

College of Wooster 
Coffee Break 

"Thd Process of Preparing Programs as a Means of 

Improving Teaching," Clarence Leuba, Visiting 

Professor, Wright State Campus 

Summary of Preliminary Recommendations" and 

discussion, - Robert DeHaan 

Luncheon 

"Significance of the froject for the Future," 

Eldon Johnson, President, Great Lakes Colleges 
Association 

Mixed subgroups to discuss planq and recommenda- 
tions for further study and improvement of 
instruction 
Coffee Break 

College teams to place the ideas generated in 
mixed subgroups into a priority list. Each collego 
to choose a leader and recorder. 

Reporting session. Each college to report on its 
priority list. 

Adjournment of Conference. Each college to leave 
a set of notes with the conference chairman. 

Task force to prepare conference report and for- 
njuliite a proposal from the ideas developed during 
the afternoon 

Task force to complete formulation of report and 
proposal 

Task force adjournment 






RELATIONSHIP OP 



Progratner 

Arnold 
Brewster 
Cope 
Cook 
Davis 
Elrod 
, Farber 
(! Kebker 
.‘Kidder 
iLeuba 
Lotowski 
Marshall 
McKenna 
Montgomery 
Pelekoudas 
Schagrin 
iSchwartz 
Smith, et» al> 
Stephenson 
Van Eyl 
Wetmore 
Whiteside 

Sum of * items 

Mean times 
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taken to prepare PROGRAMED MATERIAL 
TO STUDENT TIME WORKING ON PROGRAM 

I 

Estimated Preparation / Estimated Stu^^ent 
T^imeCin weeks )40 hr. Time (in ho»r<.j 



3 

21 *' 

5 

27* 

45* 

20 * 

4 

24* 

3* 

13* 

34 

23* 

15* 

36* 

38* 

25* 

63 

16 

17* 

9 

16 * 

1-5 

307 

24 weeks 



6-7* 

Mi mm- mm 

4-8* 

8 - 20 * 

8-16* 

8 * 

6 - 8 * 

8 - 10 * 

30-40* 

4-5* 

2 - 6 * 

9* 

6-7* 

one iponth 
1-6 wieks 
12 * 



132 



10 hours 



WhiS th^atndIIn^^ of the average tinea, 

was used. ^ ^ given as a range, the midpoint 



c-\ 



•Jt 



<i j'iiiS ' aTT«4»M*ri^".uk^^ 

; 
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r 

POETRY: METHOD AND MEANING j 

Pre- or Post Test 

i 

j (Form 1) I 

ii 

I . . * 

jla The following list contains the names of figures of speech | 

often employed by poets: ‘ | 

1. Metaphor 

2. Simile 

' 3. Personification 

1 4. Apostrophe 

^ 5. Hyperbole 

6. Synechdoche 

7. Conceit 

8. Metonymy 

The passages which follow contain one or more examples of each 
of these figures* By placing the number before the appropriato 
lettern match the name of the figure with the underlined • 
example. 

A « Love bade me welcome : yet my soul drew back 
Guilty of dust and sin. 

But quick-eyed Love , obs ervi ng me grow slack 
From my first entrance in , 

Drew nearer to me , sweetly quest ioning 
If ^ lacked anything * 

B « My Shakespeare > r ise » I will not lodge thee by 
Chaucer or Spenser, or bid Beaumont lie 
a little further to make thee a room; 

C • thou art a monument without a tomb « • • • 

D * As bees 

In springt ime > when the sun with Taurus rides, 
pour forth their populous youth about the hive 
In clusters « « « / so thick the airy crowd 
swarmed* . . « 

E * She is all states, and all princes, I. 

Nothing else is. , 

Princes do but play usj compared to this. 

All honor^'s mimic, all wealth alchemy. 
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F. I can l©ve both fair and brown, 

Her whom abundance melts, and her whom want betrays, 

Her who loves loneness best, and her who mocks and plays, 
Her whom the country formed, and whoa the town. 

Her who still weeps with spongy eye s , 

And her who Is dry cork , and never cries ; 

I can love her, and her, and you, and you| 

I can love any, so she be not true* 

Pre-Test 2 







G. 



H. 



I start out of my sleep to think 
Some day I may forget 
Their* "food and drink; 

©r, the house door left unshut. 

The hare may run until it*s found 

The horn's sweet note and tooth of hound. 



*pet animals: a hare and a cat*. 




(The following is a conversation among t)iree kings*) 



1st K. "What say you, France ? 
2nd K. " England , I say nayl^* 
1st K. "And you, Bohemia ?" 

3rd K. "Bohemia votes yea I" 



In your own words, define and give two or three example of 
literary convention. 



Pre Test 3 



Pre-Test -- Image and Symbol 
jl. In your own words, state your concept of an 
2. What is a tied image? 

3* What is a free' image? 

4. What is a synaesthetic image? 



image . 




E-2 










JUL 



m 



# 






er|c 






Appendix E*1 (cont*d.) 



5. How does an image differ from a metaphor? 

6. What is a controlling image? 

7. What is a symbol? 



1 . 

2 . 



8. What is the difference between a conventional and an attribu- 
tive symbol? 



Pre-Test 5 



Analyze this poem in depth for the figures of speech it contain 



Develop a multi-level (i.e,, literal and symbolic) interpre- 
tation of the poem — an interpretation that is self-consisten” 
and is consistent with the text.* 



The Mother of God 



by 



William Butler Yeats 



s. 



The threefold terror of love; a fallen fl^re 
Through the hollow of an ear; 

Wings beating about' the room; 

The terror of all terrors that I bore 
The Heavens in my womb. 






Had I not found content among the shows 
Every common woman knows. 

Chimney corner, garden walk 



Or rocky cistern where we tread the 
clothes 

And gather all the talk? 



What is this flesh I purchased with 
my pains. 

The fallen star my milk sustains 
This love that makes my heart's blood stop 
Or strikes a sudden chill into my bones 
And bids my hair stand up? 
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THE LANGUAGE OF LOGIC . Form 1 

Which of the following is commonly used as a logical symbol? 

A. < B . V C. ■ST' D, III 

Which of the following is not a logical symbol? 

A.<— B. /\ C. —I D. 

Select the formula that correctly symbolizes: If the record is 
cracked then the stylus skips and the sound is not pleasant* 

A. (R.— ^s) V— iP c. R:i^(S./^P) 

B. RsssSe-P D. S)/\^r— P 

Select the English sentence that * correctly Interprets: (F.S*) 
v^P F: the fever subsides; S: the spots disappear; 

P: the patient recovers* 

A* If the fever subsides and the spots disappear then the 
patient recovers. 

B. The patient will recover only if the fever subsides and the 
spots disappear. 

C. Neither the patient recovers nor does the fever subside or 
the spots disappear. 

D. Either the fever subsides and the spots disappear, or the 
patient doesn't recover. 

Select the formula that correctly symbolizes: If Jones was not 
lying,,. _he_ei^er saw Smith hit Kelly but did not believe 

Smith s blow w'a^ the cause of Kelly's death, or he did not see 
Smith hit Kelly. 



A. (-J-^S) 6 (-B V -S) C. -( J ( Sr> -B) V -S)) 

B. » ( (S. B) v^S) D. ^ J 3 (S A ~ B A-~S) 

Find the English sentence that correctly interprets: (A*B)3 

(FvC) A: Able is elected; B: Baker is defeated; C: the city 

vote was light; F: our fears were unfounded. 

A. If Able 18 elected and Baker defeated, then either our fears 
were unfounded or the city vote was light. 

Be If Able is elected then Baker is defeated, and, furthermore, 
our fears were unfounded and the <;ity vote was light. 

C. Only if Able is elected and Baker defeated will it be the 
case that our fears were unfounded or else the city vote 
was light. 

D. If Able is elected then Baker is defeated, or our fears 
were unfounded and the city vote was light. 
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7. How many ilqwo would the truth table for this formula have? 

(p. („q V ^s)).(r PZ> q)) 

A* 6 B. 8 C. 16 D. 24 



8 . 



9. 



1 . 



2 . 



In the truth table for the following formula* how many of the 
rows have a false (F) entry? 

p. (/^ P !I 3 q) 

A. None B. One C. Two D. Three or more 

Which formula is logically equivalent to: p.(^^q v«-»..^p) 

C. p.(pz> q) 

D. (p v#-^q).(p V #-s^p) 



A. -^(pz> q) 

B. p v,^ (q. p) 



.0. pe(qr>r) and (p. q) (p.r ) are formulas that are: 



A. Logically equivalent 

B. Contradictory 



C. Equivalent in some cases 

D. None of the above 



Select the formula that correctly symbolizes: Some hats are 
green fedoras. 



A. (x) (Hx V Gx V Fx) 

B. Ox)(Hx . Gx . Fx) 



C. Ox)(Hx =5 (Gx . Fx)) 

D. (x)(Hx z:> (Gx . Fx) 



What is the correct interpretation of: (x)(y )(Px S> (-~Kxy^sr^ 
Uyx)) Px: X is a person; Kxy: x knows y; Uxy: x will hurt y 

A. A person doesn't always know whom he hurts. 

B. One always hurts someone he knows. 

C. Some people are hurt if they don’t know something. 

D. What a person doesn't know won't hurt him. * 

Select the formula that symbolizes: Some Republicans vote 
against any bill introduced by a Democrat. 

Rx: X is a Republican; Bk: x is bill; Dk: x is a Democrat; 
Vxy : X votes against y; Ixy: x iiitroduced y 



A 

B 

C 

D 



(x) [(Rx.Vxy) (y )(By. O z)(Dz. Izy) )] 

(3 x)[Rx. (y)(By. (3 z)(Dz.Izy)) Vxy)] 
(3 x)CRx.(y)((Vxy.By) (3 z)(Dz. lyz) )] 
(x) [Rx =5(y)((Vxyl^By).(z)(Dz ^Izy))] 
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14. Select the sentence that interprets: (x) (y)((Px.Sy.Cxy)30k) 
Px: X is a person; Sx: x is a psychiatrist; Cxy: x consults y 
Ox: X ought to have his head examined 



16 



17 



18 



19 



20 . 



A. 

B. 

C. 
D 0 



A psychiatrist ought to examine the head of anyone who 
consults him. 

Anyone who consults a psychiatrist ought to have his head 
examined. 

Anyone who ought to have his head examined consults a 
psychiatrist. 

If anyone ought to have his head examined he will consult 
a psychiatrist. 



15. Which formula symbolizes a contradiction to: (x)(Px3>t/Sx) 



A. (x)/^(Px3<vSx) 

B. (x)(Px3 Sx) 



C. Ox)(Px . Sx) 

B. (ax)/%/(Px . Sx) 



Ox ) ( /v» Fx. /x^Gx ) and (x)(Fx v Gx) are: 



A. Logically equivalent 

B. Contradictory 



C. Neither equivalent nor contra* 
dictory . 

D. Indeterminate until 'F* and 
*G* are interpreted. 



*x^(x)/x/(Bx . Rx) and O x)(Bxi:>/v/Rx) are: 

A. Logically equivalent 

B. Contradictory 



C. Neither equivalent nor contra* 
dictory. 

B. Indeterminate until 'B* and 
*R* are interpreted. 



(x)((Ax.Bx) 3 Cx) is logically equivalent to: 






A. Ox)(Ax . Bx. Cx) 

B. (x ) ( Ax Z> Bx ) 3 Cx ) 



C. (x)(Ax (Bx'^Cx)) 

D. (x)(Cx^(Ax . Bx)) 



If *Bc* mean© Cynthia is beautiful* and *Sc* means Cynthia is 
smart* what is the meaning of: 

x(Bx V Sx) 

A. Everyone is beautiful or smart. 

B. Some girls are beautiful or smart. 

C. the class of beautiful or smart things 

D. X is beautiful or x is smart 



% 



Select the correct symbolization of: All the nominees except 

Ken and John are members of a fraternity. 

Nx: X is a nominee; Rx: x is a fraternity; k: ken; j: John; 
Mxy: X is a member of y 
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21 . 



22 . 



24 



avarv 



A. (x) C(Nx. r\y ((x = j ) v (x=k))0 (3 y)(Ry.Mxy|)] 

B • ( X ) C (Nx . /X/ N j . /X/ Nk)^(y)(Ry:i>Mxy)3 

C. Ox)(3y) C(Nx.Ry) :3 (Mxy3>'x/( (x = j ) . (x=k)) )] 

Fina the correct symbolization or* There is a moment in 
persons's life after which he is no longer naive. 

Px; X is a person; Mxy: x is a moment in the life of y; 

Nxy; X is naive at time y; Axy: x is after y 

A. (3y)(x)((Px . Myx) 3 (zXAzy^/t/Nxz) ) 

B. Oy)(x)((Px . Mxy)3 (Nxy (3 z)(Azy 3>^Mxz ) ) ) 

C. (x)0 y)((Px . Myx)n> (3 z)(Azy .^Nxz)) 

D. (x)(Px 3 (a y)(Myx . ( z ) ( Azy3^ Nxz ) ) ) 

Find the correct interpretation of: (x)(y)((Jx . Sxy) Ey) 
Jx: X is a junior; Ex: x is a senior; Sxy: x speaks to y 

A. Juniors speak only to seniors. 

B. Only juniors speak to seniors. 

C. Every junior speaks to every senior. 

D. All juniors speak to a senior. 

23. Find the correct interpretation of: 3 x)(y )(x=y ) 

A. Nothing is Identical with everything. 

B. Everything is identical with something. 

C. Nothing is identical with something. 

D. It is not the case that something is identical with 
nothing. 

Find the correct interpretation of: (y)(y C I^Hx y d $(Wx v 
Bx)) Hx: X is a horse; Wx: x is white; Bx: x is black 

A. The class of horses is not a member of the class of white 

things. 

B. Every horse is not white or black. 

C. Any horse in neither white nor black. 

D. If anything is a horse it is not white or black. 

25. What relationship does j have to k? ^ 

(3 x)0 y)(3 9K)(Pxk.pyx.Pyz. Pzj. x / z . M j ) 

Pxy: X is a parent of y 

Mx: X is male 

# 

A. j is a great-grandfather of k C. j is a first cousin of 

B. j is an uncle of k 0. j it; a nephew of k 
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Appendix E-2 (cont’d.) 



i 



iNAME. 

Sdate 



The Language of Logic 
SCHOOL 



B 



_,INSTRUCTIOM^ 

D 



1 . 

(I 

2 . 

3 . 

4 . 

6 . 



e. 



GROUP. 

FORM 



INSTRUCTIONS 



Pl^ce an *X* on the dash undez 
the letter associated with the 
correct answer. 



If you wish to change your 
original choice » either erase 
carefully or ^ place a circli 
around the INCORRECT choice. 



B 



21 . 



22 . 

23 . 

24 . 

25 . 
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i 



LANGUAGE OF LOGIC ANSWER SHEET 
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Form 1 

* 






Form 


2 






B 


23. A 


’ . 1. 


A 


23. B 


■ . ■ 


2. 


A 


24. C 


2. 


C 


24. C 




|3. 


C 


25. C * 


3. 


B 


25. A 


ERRATA 

• 


r* 


D 




4. 


A 




Form 1 


5» 


B 




5. 

% 


D 




iAfl3. Answer A should have 


6. 


A 




A 


B 




a 0 between (Vxy) 








0 « 




and (y) thus: 


7. 


C 




7 . 


C 




(Rx.Vxy) (y)By. 


8 . 


C 




8 . 


B 






9. 


A 




9. 


C 




Form 2. 
• 


10. 


A 




10. 






#13. Answer A should have 


n 




D 




another parenthesis 


11. 


B 




11. 


C 




after *Dxy* Thus: 


12. 


D 




12. 


D 




( ( Sx Ly . Dxy ) ) . ■■''a-/ O z ) 


13. 


B A 




13. 


A * 




Answer B should have 


14. 


B 




14. 


C 




3 right parentheses I 
at the end 9 thus: I 


15. 


C 




15. 


D 




( Sx Dxy) ) ) 


16 . 


B 




16. 


C 






17. 


C 




17. 


A 




Form 1 AGAIN! JJ 


ifl 


C 




18. 


B 




#25. Lower case P is for- 


18, 






mula should be upper 


19. 


.C 




19. 


C 


• 


case. Thus: 


20. 


A 




20. 


A 




(Pxk.Pyx.Pyz.Pzj . 


21. 


D 




21. 


B 






22. 


A 




22. 


D 







\i 



1 



k 






. I I fi'i 

iiSiiitt«aifcai^^ 
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Appendix E-3 



(last name) 



(first name) 



(date ) 



(college ) 



jEvaluation test 



Biochemistry for Biologists 
Stephenson Sept. 1965 



Form (1 ) 



Do the best you can with this test} do not be disturbed if ’you can* 
not respond to many items. You should guess whenever you ha-ve any 
basis for doing so. 






y 



ERIC 



1. Write the atomic diagram for the magnesium atom (atomic number 

= 12 ). 



f * * 

[3. 



i*. 

5. 

6 . 



Write the structural formula for (CH^CHOHCOOH) . 

Write the structural formula for sodium acetate, 
ionic bond. H 

* 

acetic adid= 

I 

H 

Define the "hydrogen bond". 

Define chemical equilibrium for a reversible reaction. 
In the reaction sequence 



Circle the 



phosphoglyceraldehyde + dihydroxyacetone 
phosphoglyceraldehyde ;; 5 ::±:erythrose + xylulose 



fructose + 



K 

). 

9 . 



;.o. 



llhat effect would an increase in the concent rat ion of dihydrox- 
yacetone have on the concentration of erythrose? | 

Explain briefly. 

Write the complete structural formula for glucose. 

Write the structural formula for a carboxyl (acid) group. 

Diagram the general structural formula of a tripeptide. Circle 
each peptide bond. 

Describe briefly the structure of the ochelix. 
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11 . 

12. 

13. 



14. 

15. 



16. 






Define or explain the primary structure of a pr oVe inT 
What is denaturation (briefly). 

Match the following. 



^Purine 

^Sterdl 

"pyrimidine 



® NH^ 

t^Y 

rtO"V^ 



®(5 




List the three constituents of a nucleotide. 

Construct the DMA strand complementary to this DMA strand 



P-D-P-D-P-O-P-d-P-D- etc. 



I 

A 



J 



i 

A 



t 

C 



16. What IS the pH of a solution? 

17 . What is a buffer? 

Match the following: 



aldehyde 
aromatic 
ester linkage 
keto 


A» 


If H 

H-c-d 

(t ft 


^•<.H 


I. 


0 

^ « 
-0- P- 

OM 


ethyl 


F. 


O 


if 

F. 4'« 
H 


j. 


-OH 




C. 


0 

-8.-0- 


G. -a 








A 


is 

-c-c-c- 


H* -N 

X y 
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19. 

20 . 

21 . 

22 . 



24. 

25. 

26. 

27. 

28. 

A. 

B. 

C. 

D. 
C. 



for calcium ion (atomic no. = 20) and 
chloride ion (atomic no. = 17). 

Diagram a hydrogen bond between an -NHj group and a 
group. 

Define a hydrophobic bond. 

Which man first postulated 
the structure of the cchelix. 



^Which two men first postu- 
lated the helical structure 
of DMA? 



A. Beadle 

B. Crick 

C. Darwin 

D. Linneas 

E. Pasteur 



F. Pauling 
6. Schleiden 
H. Watson 



23. In the reaction: AMP + Ph + Cal.^^ ADP + HoO 

At equilibrium vill AMP or ADP be present in the greatest 
concentration? Explain your answer briefly. 

Is the forward reaction endothermic or exothermic? 

How many calories does 10 Calories = ? 



What three types of bonding are involved in enzyme - subtrati' 
binding (complexing)? 

Diagram and label the structure of a biological membrance. 

Write the general structural formula of a phospholipid. 

Write the structural formula for an ester linkage. 

To what group (acid, basic, hydroxy, S-containing , etc.) 
does each of the following amino adds belong? 



“-M-i-i-c-U-" 

H 






I I 7 

M W « 

A HihJ'ok 

^ A 

H-< 

, y t* « 

9 9 

^ •c,-. 

It * 



n 

£ 




Tt- 

6. 

H. 

I. 



Alanine 



Cysteine 



_Lysine 

^^Phenylalanine 



'•OH 
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Appendix E-3 ( cont * d . ) 

29* Match the following* 



fructose 
^ribose 
lac «:ose 



CMjC7« 

I. / V 



galactose 

^maltose ^ 

pn^ molecule with - 

linkage 

^any one pentose 
all disaccharides 



OH 
OH 




ow 



OU 

^ /--r" 



C. t^OvS«»«>H 

\( Y"** 



OH 



Of ^ \oH 

\tL 

w 

c*H 
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1 Appendix E-4 

j TEST ON THE LITERARY RELATIONSHIPS AMONG MATTHEW, MARK AND LUKE 

! 

Form Hi Choose in each question the answer which is most correct: 

1. When one compares in Matthew, Mark and Luke passages which 
narrate the same event or teaching, one discovers the need of an 
explanation for (1) while Matthew and Mark occasionally agree 
exactly, Luke never uses the same words as does Mark. (2) Only 
once is there agreement between Mark and Luke, although Matthew 

and Luke sometimes agree. (3) Matthew, Mark, and Luke can agree 
exactly. 

2. When agreement in wording occurs among Matthew, Mark, and Luke, 
the matter of explaining such agreement (1) becomes difficult since 
all the Gospels agree on the most important matters as is sh own 

jby the unanimous presentation of the last words of Jesus o^thF''''"^ 
jcross. (2) is easy since Matthew and Mark state that they are 
using Luke. (3) becomes relevant since the Gospels do not auto- 
imat ically agree even on such important matters as is seen in their 
presentation of the last words of Jesus on the cross. 



3. Agreement in- wording among Matthew, Mark, and Luke can be 
found (1) not only in the case of important words such as "Christ” 
or the names of disciples but even in such details as the numbers 
of loaves and fish in one story. (2) only with such words as one 
would expect Christian accounts to include. (3) Whenever the say- 
ings of Jesus are uttered to people vjho live in Athens. 

4. One way of discovering which Gospel was used by the other 

might be a difference in conception about theamount jof power which^ 
Jesus possessed. (1) If the Gospel assigns relatively morr^To^wer 
to Jesus, then that Gospel is earlier and would be the one used by 
the others. (2) If the Gospel assigns relatively less power to 
Jesus 5 then that Gospel is earlier and would be the one used by the 
others. (3) But Close inspection reveals that no difference can be 
found among the Gospels in their portrayal of the amount of power 
Jesus possessed. ^ 

***** 
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! Appendix E-4 (cont'd.) 






(cont'd. ) 



Look 



I 



at 

A 



the passages immediately below to 

I B 



answer questions 5 and 6 
C 



That evening, at sun- 
down they brought to him 
all whq were sick or 
possessed with demons. 
And the whole city was . 
gathered together about 
the door. And he healed 
many who were sick with 
various diseases, and 
oi|t many demons. 



cast 



Now when the 
setting, all 
had any that 
with various 
brought them 
and he laid 



sun was 
those who 
were sick 
diseases 
to him; 
his hands 



on ©very one of them 
and healed them. 



That evening they 
brought to him many 
who were possessed wi :h* 
demons; and he cast ottt 
the spirits with a woi*d 
and healed all who were 
sick. This was to ful- 
fill what was spoken 
the prophet Isaiah, 

"he took our infirmi 
ties and bore out* 
diseases . " 






5. 

( 1 ) 



The passage which 
A. (2) B. (3) C 



gives Mark's version of the incident 



IS 



selected easily lies in the fact 
that (1) B IS short; thus it is Mark, for Mark is always the 

riefest of the Gospels. (2) C is Mark's version because it tells 
^ Jesus healed («ith a wordT'O)-* is Hark because Jesus' power ’ 
to heal IS more limited here than in passages B and C. 



7. With regard to limitations on Jesus. ( 1 } Matthew, Merk, and 
Luke agree that only in power is Jesus limited. (2) Matthew, Mark 
and Luke do not agree, for Mark and Luke cap write of Jesus' de- 
clining the adjective "good" while Matthew does not share the re- 

was in? Matthew, Mark, and Luke agree that Jesus 

was not limited in any way. 



illrAAAA 






Look at the passages immediately below in order to answer question 






B 



In those days Jesus came from 
Nazareth of Galilee and was 
baptized by John in the 
Jordan. 




Then Jesus came from Galilee to 
the Jordan to John, to be bap- — 
tized by him. John would have 
prevented him, saying "I need 
to be baptized by you, and do 
you come to me?*" But Jesus an- 
swered him, "Let it be so now; 
for thus it is fitting for us 
to fulfill all righteousness," 
Then he consented. 

(Note: "fulfill all righteous- 
A.^meAna..J!jt-ei:-~^n-^ xam pi 
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Appendix E~4 (cont'd.) 

8. The passage above which gives Mark's version is (1) "A" becausC 



in this passage there is no objection to Jesus' being baptized eve i 
-• though- Jtiil s implies, a sense of sin. (2) "B" because in this 
passage the account of Jesus' baptism is expanded, and Matthew, 

^and Luke ordinarily shorten the stories of Mark. (3) "B" be- 
; cause Mark as the earlier Gospel believes that Jesus had no sig- 

|nificant limitations. The other Gospels tend to introduce limita** 

■ t ions . . 



9. The material presenting differences in Matthew, Mark, and 
I Luke in the way in which they understand limitations in Jesus 
sencourages me to think that (1) Matthew and Luke use Mark. • 
(2) Matthew and Mark use Luke. (3) Mark and Luke use Matthew. 



10. With 



^ ^ respect to the sequence of teachings (1) early Christian 
writings such as the Gospel of Thomas and Matthew, Mark, and Luke 
present a uniform sequence just as we would expect. (2) such an 
early Gospel as the Gospel of Thomas agrees with Mark, but as a 
rule Matthew, Mark, and Luke cannot agree as to the succession of 
particular teachings of Jesus. (3) there seems to be no particula:E 
sequence of teachings to which Gospels such as the Gospel of 
Thomas and Mark agree. 



events, early Christian 
agree just as they present 



111 With respect to the succession of 

writings (1) do not agree either. (2) _ ^ _ ^ 

a uniform sequence of teachings from the career of Jesus. (3) 
generally agree only after the triip of Jesus to Damascus for the 
purpose of rescuing the good Samaritan from the Woman at the Well. 



12. When Matthew and Mark are compared with John as to the se- 
quence of events with respect to such an important event as Jesus' 
driving the money-changers out of the Temple, (1) these three 
Gospels agree in the placing of the driving the money-changers out 
JtJie Temple.^ (2) there is not complete agreement among the 
three bu;^ the dXfj£^^ unimportant because they all place the 

event in the last week of Jesus' life. (3) these Gospels do not 
agree, for one places the event early in Jesus' career and the 



other two place it among the last events of Jesus' life. 



13. With repoct to the succession of events in Jesus' career ^^^he 
accounts in Matthew, Mark, and Luke (1) never agree. ( 2 ) some- 
times agree exactly, (3) agree all of the time. 



14. With respect to the succession of teachings in Jesus' careetV, 
the accounts in Matthew, Mark, and Luket(l) never agree, (2) some- 
times agree exactly. (3) agree all of the time. 
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Appendix E»4 (tont'A.^ 

Look at the materials below in order to answer question 15. 

* B _ c 



[Spying about new wine 
in ol d wine skins. 



Blasphemy against the 
Holy Spirit is 
unforgiveable. 



Blasphemy against 
H6ly Spirit is 
ujiforgiveaple. 



the 



Saying about new winq 
i_n old wine skin« . 



Saying about new wine 
in old v;ine skins. 



Those who do the will 
of God are described 
a|» Jesus* mother and 
brothers . 



The allegory of the 
w_i_cked tenants. 




!□ 




T 

ii 


le question of pay- 
ig taxes to Caesar. 







Allegory of the 
wicked tenants. 



Those who do the will 
of God are described 
ji®sus' brothers anc 
[mother. . 



Blasphemy against the 
Holy Spirit is ' 
unforgiveable .. 

— I , 

Those who do the will 
of God are described] 
as Jesus* mother and' 



I 



The allegory of the 
wieked tenants , 



The question of pay™ 
ing taxes to Caesar. 



The question of pay- 
ing taxes to Caesar. 



15. By looking at the sequence of materials in each column in 

possible to say that Matthew (1) has to be in oolumn "B" beca^L 

tbouarh"®''®" ^he sequence of sayings eve^ 

"A° oS ''B"®®irSerfc • events. (2) could L either 

agree with ’Mark”in^tho because Matthew ordinarily does not 
agree witn Mark in the sequence of materials, but there is iust 

hoean between the two. (3) could be "A" or^"C" 

°f»®" their sequences ormaJeriaj 

ft ft Aim 

16. If we were to try to answer which Gospel is being used bv th» 

e^uls In eloh r conclusions from the sequeL^^f lit- 

Luke differenonah*’- It Matthew, Mark, and 

about which ® t •'’®^f ®«<iuences to give us some deductions 

u ? ®® ‘’®t“8 by the others. (2) Matthe" Hark 

discover "I them to so that we ’cannot 

otherl m Matth!! m" Z sequence which Gospel is used by the 

Matthew, Mark, and Luke disagree so much that it 1« ' 
impossible to conclude anything. * tnat it is 






Append iK Con t * d . ) — ~ — - -- 

iu* 4 . events of the "Last Supper” are studied we found 

that Cl) Mark and Luke agree exactly on the succesion of the 
bread arid wine, as we would expect them to. (2) Luke does not 
mention the Last Supper” but Mark does. (3}i Mark and Luke do 
not agree on the sequence of the uses of bread and wine. 



I 

i 









f 
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Appendix E-4 (cont»d.) 

18. When we describe the relations of Matthew, Hark, and Luke 

and^^eLh*”®'' the sequence oi events 

and is concerned, we can generalize by saying that 

whin'^r'nW agree in sequence, Luke disagrees; 

when Luke and Mark agree in sequence, Matthew disagreed (2) 

when^^'!k^"‘^ disagree in sequence, Luke agrees with Mark 

when Luke and Mark disagree in sequence, Matthew agrees with 

(3) "J]®" JJs^^hew and Luke agree in sequence Mark disagrees; 
when Matthew and Luke disagree in sequence, Mark agrees. 



and 
when 
and 
Mark . 
and 



19. 

(1) Mark 
sequence 



arrSHs'tnL?" US to say that 



Luke's sequence sometimes, and he uses Matthew's 



at other times. 



times, and he uses Mark's 



Luke are using 
]ArtYA4rAA 



(2) Luke 
sequence 



Mark's sequence. 



uses Matthew's sequence some- 
at other times. (3) Matthew anti 



"'® °PP°®ite page in order to answer 

questions 20 and 21 . 

20. The column which gives Mark's sequence is (1) 1, (2) 2. (3) 3 

particular column is to be identified as 
giving Mark s sequence lies in the fact that (1) the first column 

or^tai?n parable of the sower and ending with the inter- 

pretation of the parable contains the items given in columns 2 and 

wi exception. (2) the second column starting with th< 

naraM*''°f against Jesus and ending with the 'interpretation of the 
parable of the sower has the items found in the first and third 
columns with only one exception. After all the Gospels of Matthew 

2°' ^<ientioal. (3) when column 2 does not agre 
with the sequence of the third column, column 1 does agree; and* 

®®'“®® sequence in the third column 

column 2 does agree; so c^olumn three is Mark. 

AeAilInllttV ,.j. ........ 

beyond the agreement which is found among 
Matthew, Mark, and Luke (1) there is a significant agreement in 

L'ike in material not found in Mark. 

(2; there is no significant agreement between Matthew and Luke. 

13; there Is a significant agreement in wording and in sequence 
between Matthew and Luke in material not found in^Mark . ^ 



E-19. 
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Appendix E“4 (cont'd.) 

j23» Several explanations can be ^^rmulated to cover the ddta re- 
vealed in comparing non-Markan material in Matthew and Luke. (1) It| 
hardly seems possible that Matthew and Luke were writing about the 
same figure. (2) Matthew and Luke are writing about the same 
figure, but they are portraying very different periods of the life 
of Jesus. One Gospel, for example, is concerned only with the last 

week of Jesus' life. (3) Matthew and Luke may have copied a source 
unknown to us. 



24. A significant fact which appears when one studies the sequences 
of non-Markan teachings in Matthew and Luke is that (1) Matthew and 
Luke never agree on placing the same non-Markan teaching in the 
sequence of Mark. (2) Matthew and Luke never agree on placing 
teachings in the sequence of Mark because Matthew and Luke do not 
share any teachings beyond the material they share with Mark. 

(3) Matthew and Luke occasionally agree on placing the same non- 
Markan teaching in the sequence of Mark. 



..I. Or how can you say 
toj your brotheri "Le*): 
me= take the speck out 
ofi your eye," when- 
thiere is the log in 
yojur own eye? You 
hypocrite, first take 
the log out of your 
own eye, an d then you 
will s<s© 

taSke the 



12 . 



ur 



f¥h 



e healing of a leper | [The heali^ of a l^er 



rfh 



clearly'^to 
speck out of 
brother's eve. 



[ The healing of a 
leper 



choosing of t~he IT] iThe choosing of the lif 






I 



The choosing of the 

12 . 



I 






l...0r how can you say 
to your brother, I 
"Brother, let me take 
out the speck that 
in your eye, "when 
you yourself do not* 
see the log that isl 
in your own eye? Yob 
hypocrite, first take 
the iDg out of your| 
own eye, and then ypu 
will see clearly tot 
take out the speck | 

that is in your ’ 
br o t - h e r - * -e - e ye» — ■ 
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Appendix E-4 (cont*d. 



B 



[wages. 



Soldiers also asked him, 
"And we, what shall we 
do?" And he said to them 
"Rob no one by violence 
or by false accusation, 
and be content with your 



Jesus is baptized ! j Jesus is baptized | [jesus is baptized 



Think not that I have 
come to abolish the law 
““d the prophetsj I 

ve come not to abolish 
t to fulfill them. 



j Consult the exhibit on the opposite page and above in order to 
; answer questions 25 and 26. 

I 

I 25. The exhibit which has nothing to do with "Q": (1) B as it 



26. The evidence on the opposite page and above shows why the 
theory of "Q" is accepted in that (1) according to exhibit B the 
same theme does not appear in Matthew as appears in Luke. (2) the 
materials of exhibits A and B show that Matthew and Luke agree in 
placing the same saying at the same point in the Markan outline. 
(3) the exhibit displays the fact that Matthew and Luke do not 
agree on the placing of the same saying at the same point in the 
Markan outline or sequence. 



examined, it is discovered that in the presentation of stresses, 
such as the attitude of Jesus toward poverty, (1) it is Matthew 
who portrays Jesus consistently but Luke is inconsistent in his 
presentation of a particular stress. (2) it is Luke who portrays 
Jesus consistently but Matthew is inconsistent in his presentatio 
of a particular stress. (3) neither Matthew nor Luke is consis- 
tent in the presentation of a particular stress. 

28. First, the placing of non-Markan sayings common to Matthew 
and Luke in the sequence of. Mark, and, secondly, the issue of the 
consistency with which a stress is presented were the grounds on 
which the theory rests that- says (1) Matthew aad Luke used a 




stands . 



j stands. (2) B if the Markan material is eliminated. (3) A as it 



27. 



When the non-Markan sayings common to Matthew and Luke are 






•4 
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Appendix E-4 (Conc*d.) 

source unknown to us. (2) Matthew used Luke in addition to Mark. 
(3) Luke used Matthew in addition to Mark. 

29. "Q” may be identified as (1) the material in Matthew and Luke 

remaining after Mark’s material and the non-Markan material common 
to Matthew and Luke are removed. (2) the unkn own sourc A^ hich 
Matthew and Luke used in addition to Mark. (3) the Markan material 
which Matthew and Luke did not use. 



s4i 






Essay Question No. 1 

Using all evidence you can remember, give in detail the reasons why 
literary afialysis of Matthew, Mark and Luke gives rise to the theory 
that Mark was the earliest of the three Gospels. 

Essay Question No. 2 

Using the evidence you can remember, give in detail the reasons why 
a second unknown source is thought to have been usdd by Matthew and 
Luke in addition to their use of Mark. 

(Si 
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I 

I 



Systematic Analysis in 
Political Science 



FORM I 



In each of the following pairs t circle the letter before that 
«i.teii^ which you believe to b^ a political inquiry* 

a. The structure of the Ohio Republican Party, from precinct 
to chairman. 

b. Factors operating to decentralize the Republican State 
Chairman's power, 

a. Membership of the General Assembly of the United Nations. 

b. Influence of the Afro-Asian bloc in General Assembly voting. 

a. The appropriate role of the pic'ess in reporting pre-trial 
events in criminal cases* 

b. Coverage of the 1964 Presidential election by the Chicago 

Tribune and New York Times . I 

Which of the following rephrasings or uses of the title, "The 
Presidium of the Communist Party (CP) of the USSR" is a poli- 
tical inquiry? Circle the letter before the correct answer. | 

a. Influence of the military upon the Presidium of the CP, ! 

b. Strategies of the Presidium of the CP in the face of 
Chinese objections to Soviet foreign policy. 

c. The uses of ideology by the Presidium of the CP in adjustijig 
to conditions unforeseen by Marx and Lenin, 

d. All of these* 

e. None of these. 

Some statements, dealing with facts, are called "descriptive," 
while others, dealing with values, are termed "normative.". 

Place a D for descriptive or an N for normative before each of 
the following: 

Human nature basically rests upon self-intereSt . 

^ * Mejj^everywhere j despite cultural differences, are pretty 
-- ^much the same, 

Q » wen naturally seek to be free and independent. 

^ • War IS inevitable, so the function of diplomacy is to 



"Ends" in men's lives, once obtained, may become "means" to ye 
higher ends* Thus, a "means" may be regarded in Dewey's ter 
as "ends-in-view. " In the following, place a 1 besides the 

• to the next highest (place 
a 2), and so on, to complete the sequence. 



stretch the time between wars 
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Appendix E-5 (cont'd.) 
4. (cont *d. ) 



5. 




cl • 

b. 



c . 

"d. 



A two-party system with competing programs and candidat 
Individualism, or the ability to live one’s life as he 
sees fit. 

Right to vote. 

Responsibility of public officials for their use of 
power. » 



e 



3 



Examine closely the following words and terms. Then, s.elect 
one of these to complete the next series of statements. Not 

ini K H A . 



W..W sei^xos OT sraLeine) 

all are usable, and some may be used more than once 



fact 

descriptive 
significance 
government 
intermediate goals 
prescript ive 



empirical 

method 

polity 

efficient 

problem 

value 



normative 

relevance 

means-end 

hierarchy 

state 

scope 



chain 



a. 

b. 



Political inquiry begins where there is 
to consider. 

Political science tends to be 



narrow in 

the focus of inquiry is merely upon the study of the 



if 



c • 

d. 

e. 



Political inquiry must contribute meaningfully to our under- 
standing of the sub j ect--that is, it must have ' 

In any scholarly discipline, there develops a 
a set of systematic procedures for learning 
An object of study which may be verified by 
called a , and statements about 



the truth* - 
observation is 
it are called 



f. 

g* 



statements . 



If the object of study deals with an assumed 

we call statements about it statements I 

A statement in the form. 

Then 



h. 

i . 



in 

called a 

A statement 



in the 
order to realize 



tt 



IS 



A statement 
is a 



in the form, 
statement . 
in the form, ”A has 
statement . 



”If you think A is a good end 
it you must use means B, 
statement . 

’’End A is a good thing,” 



IS a 



? * 
f 



1 * 

k; 



Preferential ranking of 
chain. 



two branches of government ,” 
produces a means-end 



Political science viewed as focus on the 

A ^ ^ 



as ob- 



ject of study may emphasize legal and philosophical ab- 
stracfions.” ■*” ' 
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Appendix E-5 (cont*d.) “““ 

5. (cont*d.) 

!• "What is the nature of the good society?" is not an inquiry 

that is o r , but one that 

is . 

m. One type of inquiry for political scientists is 

how to achieve a given end with minimum cost; this kind of 
inquiry deals with the question of what is • 

~ ^ —a— 

"The Presidential vote from 1944 to 1964 in the center city, 
suburbs, and non-metropolitan areas of the U«S." are data. 

Pose an empirical, prescriptive, and normative inquiry about 
them in one sentence each: 

I 

a. Empirical: 

b. Prescriptive: 

c. normative:' 

In scientific methodology, whether in the natural or social 
sciences, the terms "theory," assumption," And "hypothesis" 
are often used. In your own words, define them (one sentence 
each): 

a. Theory: 

b. Assumption: 

c. Hypothesis: 



8. Theory generates testable 



, it helps to 



^existing facts or events, and if valid, it may be used to 
future events. 



_a. 

^b. 

d. 



Indicate whether each of the following is a theory, assumption, 
or hypothesis by placing to the left of the item a T, A or H. 

In international relations, nation-states are motivated 
purely by consideratiqns of their national interest. 
States seek to achieve power which is a least, and no- . 
less than', that of other states. 

Failure of a state to signal clearly to other states its 
intentions or policies works against the achievements 
of those policies. 

Strong support given by a population for its leaders^' 
foreign policies tends to make difficult any effort^ by 
the leaders to change those policies. ' — 

War is an expression of man's state of sin. 
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Appendix E-S (cont’d.) 



10 . 



11 . 



12 



In political science there are three major "approaches?*' 

Institutional--!. e. , study of stable patterns of group 

behavior . 

^ * Philosophical-- i . e . , ethics t values, etc. as the cen*^ 

tral concern. 

Behavioral--!. e. , analysis of interactions of individufeils . 

Place a 1 before the approach you believe is the oldest, a 3 
before the newest, and a 2 befdre the approach of intermediats 
age. 

Determine which of the three approaches briefly defined in 
#10 is most appropriately associated with each of the follow- 
ing (place an I^, P or ^ before each item). 



3 • 



b. 



Analysis of the liberal and consei'vative members on 
the Supreme Court on the question of a defendant's 
rights in a criminal -trial. 

Analysis of the righ:ts of a defendant in a criminal 
case as defined by the written opinions of the Supreme 
Court. 

Analysis of the proper balance between the rights of 
a defendant and the rights of the community to pro- 
tection and order. 



d. 

[e. 

f. 



Associated with internal consistency. 

Associated with systematic description of formal 
structures leading to generalizations. 

Associated with emphasis upon reliable methodology. 



Most characteristically would deal with: 

_g . -'Render unto Caesar that which is Caesar's and unto 
God that which is God's. 

^h. Voter motivations in 
i. A high school civics 



the 1964 election, 
textbook. 



Most characterictic weakness would be: 

_ j . Tends to omit description of structures. 

_k. Fails to relate ideal to real behavior. 

^1. Although descriptive, fails to account for the indi- 
vidual's actions. 

Three "theories" important in political science today are: 

” Power Theory: explains a relationship among persons in 
which one person induces another to act differently from 
his normal pattern of behavior. 



a « 
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Apj>endix E-S ( c o n t ‘Tf) 

12. (cont*d. ) 

b. Group Theoryi explains politics as a process resulting from 
the conflict of human organizations. 

c. Decision-Making Theory! explains how a choice of action Is 
determined. 

Estimate for which theory the statements below are most appro- 
priate and place an a, b, or £ to the left of each. Some 
statements are appropriate for more than one theory; indicate 
this by placing more than one letter in the space providted. 

X 

a . Deals with a quality characterized as having the threat 
of sanctions. 

b . Incomplete as a theory and appropriate for some problems 
but not for others. 

c/ Studies ”elitcs"--i. e . 9 a minority whose views seem to 
prevail in most situations where conflict exists.-- 

d . More than the others deals with a "process"--i.e. , move- 
ment, activity, relationship among events and actors. 

e. The hypotheses it generates are not fully testable or 
always significant. 

f. Usually focuses upon a relationship traditionally recog- 
nized as central in the political process. 
g» Its central concept may be defined as shared attitudes 
. on the basis of which certain claims are made on others 
having different attitudes. 

h. ' May employ simulation devices, such as game theory. 

i. Utilizes the case study as a technique of analysis, 

although this technique lends itself poorly to gejriej?.aXiz 
zation. '' ' ' 

j. Would deal with such subjects as an active minority, 
overlapping membership, cohesion, and access to public 
officials. 

k. Deals with such terms as "influence” and "authority.” I 
'l. Would best be employed in studying a lobby organization. 
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Student Data Roster (Do not distribute to students) 
and Teaclilng-Mc'thod Evaluation Questionnaire 



Th^ following information about each student should be filled in 
^forder to evaluate the results of the various design experiments with each 
other, the personal student data is absolutely necessary. All intoima ion on 
this sheet will be kept in strictest confidence. Please print. 



Design Data 



Program Title_ 



Evaluator's Name 

Design Numbi r 

Group Number 



Student's Name_ 



Sex 



Last Name First 

(cc 21) Class^ 



CEEB Scores 

Math 



(cc 23-25) 



Verbal 



Test Form 2 (Pre Test) 



Text Form 2 (Post Test) _ 



Test Form 1-B (Final or Retention Test_ 



Content test No. p.g ns,, 
Content test No. 2) 



Rank in high school graduating class_ 
Size of high school graduating class_ 



Cumulative GPA in college (omit if 

student if a beginning freshman)_ 






(Card No. 1) 

_(cc 1) 

(cc 2) 

(cc 3) 

(cc 4) 



(cc 5-20) 
(cc 22 ) 



(cc 26-28) 
(cc 29-31) 
(cc 32-34) 
(cc 35-37) 



(cc 38-39) 

(cc 42-44) 
(cc 45-47) 



(cc 48-50) 
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Tc-aching-Mothod Evaluation Questionnaire 
(To be filled in by student) 



(Card No. 2) 



As you may know, you have been par(*icipating in a study of the effectiveness of 
various methods of presenting subject matter to students. Will you kindly give your 
reactions to the methods used in your course or section? This questionnaire is to 
be used for research purposes only and* will not affect your grade. Please print. 



1. Student Name 



(cc 5-29) Class 



Last Name First 



Course 



Instructor 



College 



Estimate to the nearest hour the total amount of time you spent outside of class 
during the experiment in this course. Itemize the time into these categories; 



Working on programed material (cc 23-24) 



Reading text material and/or working assigned problems 

(cc 25-26) 



_Re vie wing (cc 27-28) 



During this experiment, what materials and/or methods were used to present subiect 
matter to you? Check one or more categories below: 



Programed Material (cc 29) 
_Text Material (cc *30) 



Lectures (cc 31) 
_Dlscussion (cc 32) 



4. To the left below you va/III see four columns headed by the words programed material, 
text material , etc. On the right below are five evaluative stateiiients in 1 ine 
with the blank spaces in the columns. Place a check in the column under each 
method or materials you used in line with the statement on the right that best fits 
your evaluation of it, omitting columns that do not apply to you. You may use the 
same evaluative statement for moie than one method if you need lo. 



Programed 

Material 



Text 

Materia 1 



Lectures Discussion 



_Very positive learning experience (1) 



More positive than negative (2) 
Indifferent, no strong reaction (3) 



More Negative than positive (4) 
Very poor learning experience (5) 



5. Evaluate the pace or the rate at which nev; information was presented to you by 
each of the im.^thods or materials listed in the columns below on the left. 
Programed Text 

Material Lecture Discussion 

(cc 38) (22 39) (cc 40) 



Material 
(cc 37) 



The pace was too fast (1) 

_The pace was about right most of the 

t ime ( 2 ) 



jrhe pace was too slow(3) 
The pace was uneven (4) 
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6. Did you have a senst- of direction, a feeling of getting some place, 
P r o g r a me d T e x L 

Material Material Lecture 

(cc 41) (cc 421 



(ce 43) 



Discussion 
(cc 44) 



Ycs, I xvas never in doubt (1) 



Most of the time (2) 
S ome t irne s (3 ) 



Usually not (4) 

No, I was lost mos.t of the time (5) 



7. 



Could you review the material easily? 
programed Text 

Materia 1 
(cc 46) 



Material 
(cc 45) 



Lecture 
(cc 47) 



Di scuss ion 
(cc 48) 



Very easy to revie\^; (1) 
Fairlv easv to reviev>; (2) 



Fairly 

/ Difficult to review (3) 



Very difficult to review (4) 
Did not revievr (5) 



8 . 



Rank the ’"ollowing methods of being presented with information or subject matter 
according to your preference. Give the rank of 1 to your highest preference, 

2 to your next highest, etc., until you have ranked all methods you used in this 
study . 

^Programed Material (cc 49) 



Text Material (cc 50) 
Lecture (cc 51) 
Discussion (cc 52) 



before this experiment? (cc 53) 

^My most like subject (1) 



I disliked Che subject more than many other subjects (4) 



My least liked subject (5) 



10. How have the methods used in the experiment affected your attitude toward this 
subject? 

Programed Text 

Material Material Lecture Discussion 

(cc 54) (cc 55) (cc 56) (cc 57) 

Favorably (1) 

^No appreciable effect (2) 

Unfavorably (3) 
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11 rIiow are four pairs of apposite iXds with five spaces between. You are to . 

“• !:a“atrth: LthocU used"m your group Cor tbi.s experiment. Indicate your^ 

attitude by checking In the appropr iat.e space between each o . P 

acticuue > b would indicate you thought it was 

ot „ould inuicate you thought it was stimulating. 

r'cne?k in Ihe second space from the right would indicate you thought it somewhat 

stirulatingf etc. Evaluate all the meth.ids in which you participated. 



p r ogi' ame d Ma t e i* i a 1 



Boring 



Stimulating (cc 581 



1 !( 



I « 



Too hard 

Efficient 

Delightful 



Text Material. 



Too easy (cc 59) 
Inefficient (cc oO) 
Irritating (cc 61) 



Boring 
Too liard 



Efficient 



Delightful 



Stimulating (cc 62) I 

Too easy (cc 63) [ 

Inefficient (cc 64) ^ 

i 

Irritating (cc 65) t; 



Lecture- 



Boring 
ioo hard 
Efficient 
Delightful 



Stimulating (cc 66) 



Too easy (cc 67) 
Inefficient (cc 68) 



Irritating (cc 69) 



Discussion 



Boring 

Too hard 

Efficient 

Delightful 

12. Have you ever used programed material 
Where did you use it? (cc 75) 



Stimulating (cc 70) 
Too easv (cc 71) 
Inefficient (cc 72) 
Irritating (cc 73) 

before this time ;c'cs ^No (cc 74) 

In junior High School (1) 

In Senior High School (2) 

In College (3) 

At Home (4) 



What subject matter v;as covered by programed materials? 























^ f>e 



Table 1 Composite Student Attitudes for all Students 



Variable* 


Statistic 


Program 


Text 


Lecture 


Discussion 


W 

Positive 

Evaluation 


Mean Rank* 
N 


1.95 

831 


2.33 

328 


1.86 

602 


2.15 

769 


Q5 

Pace 


Mean Rank*** 
N 


2.10 

831 


2.13 

311 


1.98 

603 


2.21 

722 


Q6 

Direction 


Mean Rank^'f* 
N 


2.20 

832 


2.38 

321 


2.10 

601 


2.44 

738 


Q7 

Review 


Mean Rank** 
N 


2.14 

828 


2.31 

325 


2.04 

598 


2.92 

670 



Appendix for exact wording of Questions 4^ 5> 6 , 7 . 

**The lower the numerical mean rank on these questions, the more favorable 
was the rating; 3 x\ras the midpoint of the scale. 

"ii'k'kPi response of 1 indicated the pace was too fast. 

A response of 2 indicated the pace was about right. 

A response of 3 indicated the pace was too slow. 

A response of 4 indicated the pace was uneven. 





Table 2 


Student Attitude 


IS toward Poetry Unit 




Variable 


Statistic 


Program 


Text 


Lecture 


Discussion 


Learning 


Mean Rank 
N 


2.14 

152 


2.16 

99 


1.79 

131 


1.80 

124 


pace 


Mean Rank 
N 


2.02 

153 


2.10 

93 


2.18 

128 


2.19 

118 


Direction 


Mean Rank 
N 


2.22 

153 


2.31 

95 


2.00 

129 


2.14 

123 


Review 


Mean Rank 
N 


1.81 

152 


2.02 

96 


2.00 

128 


2.53 

119 






















Table 3 Student Attitudes toward Logic Unit 



Variable 


Statistic 


Program 


Text 


Lecture 


Discussion 


Learning 


Mean Rank 


1.91 


2.36 


2.11 


2.38 




N 


199 


81 


89 


139 


Pace 


Mean Rardc 


2.35 


2.20 


1.98 


2.33 




N 


198 


79 


92 


114 


Direction 


Mean Rank 


2.21 


2.35 


2.24 


2.62 




N 


198 


81 


90 


117 


Review 


Mean Rank 


2.67 


2.54 


2.49 


3.36 




N 


198 


80 


86 


90 


k 


Table 


i 4 Student 


Attitudes 


Toward Biochemistry Unit 


Variable 


Statistic 


Program 


Text 


Lecture 


Discussion 


Learning 


Mean Rank 


1.86 


2.66 


1.90 


2.36 




N 


217 


100 


179 


294 


Pace 


Mean Rank 


1.81 


2.15 


1.86 


2.11 




N 


218 


95 


179 


284 


Direction 


Mean Rank 


2.28 


2.67 


2.22 


2.54 




N 


218 


99 


178 


287 


Review 


Mean Rank 


2,18 


2.63 


2.15 


2.76 




N 


216 


103 


179 


269 
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Table 5 Student Attitudes Toward Religion Unit 



. 

ERIC 



Variable 


Statistic 


Program 


Text 


Lecture 


Discussion 


Learning 


Mean Rank 
N 


1.77 

155 


1.95 

40 


1.70 

132 


1,82 

142 


Pace 


Mean Rank 
N 


2.31 

155 


2.05 

37 


1.98 

133 


2.24 

138 


Direction 


Mean Rank 
N 


1.87 

155 


1.89 

38 


1.90 

132 


2.33 

141 


Review 


Mean Rank 
N 


1 . 64 
155 


1.79 

39 


1.65 

133 


3.19 
' 128 




Table 6 Student 


Attitude 


Toward Political 


Science Unit 


Variable 


Statistic 


Program 




Lecture 


Discussion 


Learning 


Mean Rank 
N 


2.21 

108 




1.92 

71 


2.10 

70 


Pace 


Mean Rank 
N 


2.01 

107 




1.92 

71 


2.41 

68 


Direction 


Mean Rank 
N 


2.44 

108 




2.21 

72 


2.49 

70 


Review 


Mean Rank 
N 


2.24 

107 




1.99 

72 


3.14 

64 
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Tables for Individual Studies 

EXPLANATION OF THE ANALYSIS OF VARIANCE 
The analysis of variance is based on repeated measures on the sauie 
subjects (pre and post test scores) In analyzing the studies under 
Design #2 three measures were compared on each subject: pro, post, and final 
test scores. This was done because the differences in methods of instruction 
under Design #2 took place between the post tests and the final tests. 

The colvimns headed source : source of the variability in the test 
score; S^: sums of squares; degrees of freedom; MS: mean squares; and 

F: f-ratio are all used in the computation of the final f-ratio. The F 
statistic in the row labeled "learning” indicates^!# there is a significant 
difference in the meaner scores on pre tests and post tests which can be 
used as a basis for concluding that learning has taken place. The f-ratio 
in the rovj labeled ’l^oups" indicates if there is any significant difference 
in the amount of learning due to the difference in methods of instruction 
that the groups received. The final f-ratio in the rov; labeled ’.'interaction” 
indicates whether or not there was a significant difference in the rate of 
learning of the groups being compared. 
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Study 1: Miaer-Johnson - Design #1 

Table la -Means and Standard Deviations 



Group 




Pretest 


Posttest 


Program 


M 


31.92 


73.75 




SD 


11.37 


11.80 




N 


24 




Lecture 


M 


18.19 


56.46 




SD 


11.33 


16.35 




N 


26 





Table lb- Analysis of Variance - Program Group - Mizer 





Table Ic - Lecture 


Group - Johnson 




Source 


SS 


df 


MS 


F 


Between Subjects 


5,519.98 


23 






Within Subjects 

Lea rning(Pre -Post) 
Residua 1 


22,586.50 

19,723.52 

2,862.98 


24 

1 

23 


19,723.52 

124.48 


158.45*** 


Total 


28,106.48 


47 
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Source 


SS 


df 


MS 


F 1 


Between Subjects 


4,426.67 


23 




5 

d 

f 


Within Subjects 


22,748.00 


24 




1 

1 


Learn ing(Pre -Post) 


21,000.34 


1 


21,000.34 


276.36^’«v* • 

t 


Residua 1 


1,747.66 


23 


75.99 


g 


Total 


27,174.67 


47 
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study 2: (Church - Design*#!) 



Group 



Program 



Lecture 



M 

SD 

N 

M 

SD 

N 



pretest 


Posttest 


20.83 


69.88 


13.18 


13.09 


24 




25.13 


72.39 


16.06 


7.88 


23 





Source - 

Between Subjects 
Groups 

Error Between 
Within Subjects 

Learning(Pre-Post) 

Interaction 
TestsfXSs w/in gps 



Table 2h - Analysis o£ Varianjce 

df MS 



1.0 

44.9 



1,0 

1.0 

44.9 



272.64 

187.29 



54461.14 

] S . ».n 
U2. M 



1.46 



383.27-' 

.13 




Program 



Lecture 



M 

SD 

N 



55.50 

13.54 

12 



69.17 

9.16 



Source 

Between Subjects 
Groups 

Error Between 

Within Subjects 

Learning (Pre-Post) 
Interaction 



df 


MS 


r 


1.0 


39.22 


.17 


20.9 


236.62 




1.0 


1534.05 


8.91-’^* 


1.0 


50.92 


.30 


.20.9 


172.12 





Total 






***p<;;^ .001 



*p< .05 



**p< .01 
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study 4: Burkett - Design* #2 



Table 4a 


Means of 


Groups 




Group 


Pretest- 


T| Posttest 


-T., Final Test-T„ 

2 3 


Program-Discussion 


M 19.91 


55.36 


50.95 




N 22 






Program-Lecture 


M 23.37 


44.42 


43.37 




N 19 






Table 4b 


Analysis 


of Variance 




Source 


SS 


df 


MS F 


Betvjeen Subjects 


14,197.02 


40 




Groups 


470.45 


1 


470.45 1.34 


Error Between 


13,726.57 


39 


351.96 


Within Subjects 


35,606.00 


82 




tests 


22,473.25 


2 


11,236.63 70.37**' 


interaction 


677.20 


2 


338.60 2.12 


tests XSs w/in gps 


12,455.55 


78 


159.69 


Total 


49,803.02 


122 




Individual comparisons 






Differences between Tj^ - T^ 


and Tt - T 3 i 


are significant. 


Difference between T 7 - T-i*" 


is not significant. 


Ti = 21.17 


T, = 52.44 T, 


= 47.44 


*P< .05 


**p< . 


01 ***.vp< .001 


Study 5: Boyle - Design #3 






Table 5a 


Means and Standard Deviations 


Group 




Pretest 


Posttest 


Progr am- Lecture 


M 


15.53 


29.37 




SD 


6.45 


6.22 




N 


30 




Lecture -Program 


M 


17.74 


29.26 




SD 


5.47 


9.17 




N 






Table 5b 


Analysis 


of Variance 




Source 




df 


MS F 


Between Subjects 








Groups 




1.0 


28.71 .39 


Error Between 




50.0 


74.00 


Within Subjects 








Learning (Pre -Post) 




1.0 


4,184.78 107.66^ 


Interaction 




1.0 


.90 


Tests XSs w/in gps 




50.0 




Total 




103.1 




*P < . 05 


**P< .01 


.001 
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study 6: Anderson •» Design 

Table 6a - Means and Standard Deviations 



Group 




Pretest 


Posttest 


Program 


M 


3.S4 


43.36 




SD 


8.32 


18.25 


Text 


M 


4.77 


42.33 




SD 


6.03 


15,93 




N 


29 





Table 6b - 


Analysis of Variance 






Source 


df 


MS 


F 


Betv\?een Subjects 








Groups 


1.0 


.07 


.00 


Error Between 


52.3 


205.02 




Within Subjects 








Learning (Pre-Post) 


1.0 


40516.05 


268 . 35*''«v* 


Interac: 1 *n 


1.0 


25.01 


.17 


Tests XSs r/in v b. 


52.5 


150.93 




Total 


108.0 









Study 7: 


Angel 1 - Design fM 
Table 7a - Means 


and Standard 


Deviations 


Group 




Pretest 


Posttest 


Program 


M 


10.26 


57.17 




SD 


11.16 


17.06 




N 


23 




Text 


M 


9.58 


49.74 




SD 


14.65 


11.51 




N 


19 







Table 7b - 


Analysis of Variance 






Source 




df 


MS 


F 



Between Subjects 



Groups 


1.0 


342 . 94 


1.28 


Error Betv^7een 


39.6 


267.37 




Within Subjects 








Learning (Pre-Post) 


1.0 


39441.05 


207.09*** 


Interaction 


1.0 


237.41 


1.25 


Tests XSs w/ in Gps. 


40.7 


190.45 




Total 


82.2 







***?< .001 



er|c 



*P< .05 



**P< .01 















study 8: 


Van Eyl - Design^T^ 
Table 8a - Means and 


Standard 


Deviations 




Group 




Pretest 


Posttest 




Program 


M 


13.55 


42.00 






SD 


14.61 


19.57 






N 


29 






Text 


M 


13.52 


31.45 






SD 


9.27 


15.39 






N 


33 




• 



Table 8b - 


Analysis c 


>f Variance 






Source 




df 


MS 


F 


Between Subjects 










Groups 




1.0 


867.62 


2.8 


Error Between 




60.0 


309.72 




Within Subjects 










Learning (Pre-Post) 




1 .0 


16676.17 


111.18*** 


Interaction 




1.0 


857.05 


5.71* 


Tests XSs w/in gps. 




60.0 


150.00 




Total 




123.0 







Study 9: DeHaan - Design #2 



Table 9a - Means of Groups 



Group 




PretesC-Tx 


Posttest-T2 


Final Test- 


Program-Discussion 


M 


4.74 


34.11 


42.31 


N 


35 






Program-Lecture 


M 


4.93 


43.46 


44.54 


N 


28 







Table 9b - 


Ana lysis 


of Variance 






Source 


SS 


df 


MS 


F 


Between Subjects 


16521.19 


62 






Groups 


718.10 


1 


718.10 


2.77 


Error between 


15803.09 


61 


259.07 




Within Subjects 


71582.87 


126 






tests 


55219.61 


2 


27609.81 


215.31*** 


interaction 


719.03 


9 


359.52 


2.80 


tests XSs w/in gps. 


15644.23 


122 


128.23 




Total 


88104.06 


188 







Individual comparisons: 

Differences bet^'een - T9, Ti - Tj, T2 ■ 9 ^® significant. 

T, ^ 4.82 , T2 = 38T27, 13 == 43.30 

*P< .05 " **P< .01 **Vcp< .001 
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study 10; 



S chagrin - Design #3 



T n b 1 c lOci 


-Means and 


Standard Deviations 


Group 






Pretest 


Posttest 


Program then Lecture 


M 




7.33 


42.93 




SD 




5.91 


11.83 




N 




15 




Lecture tlien Program 


M 




11.60 


49.87 




SD 




10.06 


13.34 




N 




15 




Table 10b- 


- Analysis 


of Variance 




Source 


df 




MS 


F 


Betx\?een Subjects 










Groups 


1.0 




470.40 


4.00 


Error Between 


28.0 




117.73 




Within Subjects 










Learning (Pre-Post) 


1.0 




20461.07 


187.1 5*** 


Interaction 


1 . 0 




26.66 


.24 


tests XSs w/in gps . 


28.0 




109.33 




Total 


59.0 








Study 11; Tovo - Design #4 








Table Ha* 


“ Means and 


Standard Deviations 


Group 






Pretest 


Posttest 


Program 


M 




16.19 


52.58 


(Volunteers) 


SD 




16.04 


18.48 




N 




26 




Lecture 


M 




14.91 


43.14 


(Volunteers) 


SD 




14.55 


14.21 




N 




23 




Table ilb 


- Analysis 


of Variance 




Source 


df 




MS 


F 


i^CL'V'/een Subjects 










Groups 


1.0 




685.21 


1.82 


Error Beti^een 


45.6 




377.18 




Within Subjects 










Learning (Pre-Post) 


1.0 




24874.23 


123.86'«f-'V 


Interaction 


1.0 




396.49 


1.97 


Tests XSs w/in gps. 


45.6 




200.82 




Total 


94.3 








*P< .05 




**P< .01 


.001 
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Study 12: StcphenHon and Wilcox - Design #1 



t 



If X 



! 






Group 


1. #5* lb. 4 i V4440 


Pretest 


Posttest 


Program 


M 


6 . 84 


30.29 




SD 


6.29 


18.38 




N 


31 




Ie:;t 


M 


8.43 


30.68 




SD 


5.97 


12.15 




N 


28 




Lecture 


M 


5.29 


34.47 




SD 


7.11 


15.62 




N 


34 




Table 


12b- Analysis 


of Variance 




Source 


df 


MS 


F 


Betvv'een Sitbjects 








Groups 


2.0 


1 608 . 94 


8 . 54** 


Error Between 


89.4 


188.38 




Within Subjects 








Learning (Pre-Post) 1.0 


46218.28 


386.48*** 


Interaction 


2.0 


1800.25 


15.05*** 


Tests XSk w/in gps. 89.4 


119.59 




Total 


183. 8 






Study 13: Yov-j-Design #1 






Tab le 


I3a- Means and 


Standard Dev 


lationr 


Group 




Pretest 


Post test 


Program 


M 


19.68 


51 .04 




SD 


12.56 


18.72 




N 


27 




Lee t ure 


M 


18.84 


25.09 




SD 


11.05 


9.30 




M 


30 




Tab le 


13b- Analysis 


of Variance 




Source 


df 


MS 


F 


Be t VK'CMi Sob jec t s? 








Groups 


1.0 


5033 . 70 


23.19*** 


Error Betun^en 


54.1 


217.09 




Within Subjects 








Learn Ing (Pro -pos t ) 1 . 0 


9926.40 


283.31*** 


Interaction 


1 .0 


4424.65 


126.28*** 


T’st.s XSs vj/in 


gp.«s . 54.1 


35.04 




Total 


in . 2 


t 
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m 



M 






S1iaJg:iPj5i3#?®SSSiaSS;SW-a,si«fc::;^-^ 



o 

ERIC 




.. . • 



Stiuty 14: Stephenson and Wilcox 

Table 14a- Moans of Croups 



Group 



(Mixed) 



N 



N 



N 



Pretest-T^ 


posttest-T-j 


Final Test-T3 


8.96 


60.56 


51.96 


27 






5.25 


46.03 


42 . 36 


28 






5.82 


57 . 65 


47.88 


17 













Table I4b - Analysis of Variance 



MS 









ERIC 



Source 

Betvv-een Subjects 
Groups 

Error Between 


55,897.09 

2,808.16 

53,088.93 


71 

69 


1404.08 

769.40 


1.32 


Within Subjects 
tests 

interaction 
tests XSs w/in gps. 


111,360.00 

90,973.80 

1,103.21 

19,282.99 


144 

2 

4 

138 


45486.90 

275.80 


325.53*^v* 

1.97 


Total 


167.257.09 


21 




— . 



inaiviauai ijomp«e»i 

Differences between Ti-T,, Ti'T,, Tj-Tj arc signUicant 
■r, = 6.67. T, = 65.75 T3 = 4V.40 

*P ^ .*15 ” ^ .*11 ***..'P 4 .001 

Study 15: atton - Design #3 

Table I 5a- Means and Standard Deviations 



Groups 



Pretest 



Post test 



program then Lecture 



M 
SD 
N 

Lecture then Program M 

SD 

I'l 



4.96 

6.65 

28 

7.33 

8.88 

24 



74.54 

16.29 

74.08 

17.93 



Source 



Table 15b- Analysis o £ Variance 

MS 



df 



Betv.’een Subjects 
Groups 

Error Between 

Within Subjects 

Learn ing(Pre -Post) 
Interaction 
Tests XSs v;/in gps 
To ca 1 



1.0 


23.75 


49.6 


241.09 


1.0 


120078.20 


1 .0 


51.43 


49.6 


148.4 


102.3 





.10 



809.15'W«V 

.35 



*P< .05 



**p< .01 



**;«vp< .001 
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Studv 16: Wilson - Design #4 



Group 



Program 



Lecture 





Pretest 


Posttest 


M 


6.23 


71.70 


SD 


6.64 


14.73 


N 


30 




M 


8.21 


59.30 


SD 


7. o5 


19.42 


N 


33 





Table 16b“ Analysis of Variance 



Source 



df 



Betvx»een Subjects 
Groups 

Error Between 

Within Subjects 

Learning (Pre-Post) 

Interaction 

Tests XSs w/in gps. 

Total 



MS 



F 



Group 



Program 



Lecture 





1.0 


852.80 3.91 




60.8 


218.17 




1.0 


106744.53 1004.40*^v* 




1.0 


1623.79 15.28*'«*f* 




60.3 


106.28 




124.7 




Design #1 
■ Means and 


Standard 


Deviations 


Pretest Posttest 


M 


2.20 


71.15 


SD 


3.96 


12.29 


N 


20 




M 


1.84 


62.32 


SD 


1.61 


11.09 


N 


19 






Table 17b- 


Analysis of 


Vai iance 




Source 


df 


MS 


F 


Between Subjects 








Groups 


l.O 


411.65 


4 . 7 6* 


Error Between 


36.9 


86 .48 




Within Subjects 








Learning (Pre-Pose) 


l.O 


81604.93 


1028 . 38*** 


Interaction 


1.0 


350.07 


4.41* 


Tests SXs w/in gps. 


36.9 


79.35 




Total 


76.9 






*P< .05 




.01 


***p^ .001 
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Study 18: Stegner - Design #1 

Table 18a- Means and Standard Deviatior 



GiT oup 




Pretest 


osttest 


Program 


M 


14.76 


65 . 66 




SD 


7.90 


15.92 




N 


29 - 




Lecture 


M 


14.07 


60.80 




SD 


9.39 


13.11 




N 


30 




Table 


18b- Analysis 


of Variance 




Source 


df 


MS 


V 


Between Subjects 








Groups 


1.0 


226.88 


1.44 


Error Between 


56.9 


157.22 




Within Subjects 








Learning (Pre-Post) 1.0 


70275.33 


558 . 20*** 


Interaction 


1.0 


127.77 


1 .01 


Tests X Ss \<i/in 


gps 56.9 


125.90 




Total 


116.9 







Study 19: Scott - Design #2 

Table 19a- Means of Groups 
Pretest-Tj^ 



Group 



Posttest-T. 



Final Test-T, 



Program-Discussion M 26.36 

N 22 



77.14 



77.77 



£ 



m. 



Program-Lecture 


M 15.54 
N 27 


78.00 


77.00 




Table 


19b- Analysis 


of Variance 






Source 


SS 


df 


MS 


F 


Between Subjects 


3878.59 


47 






Groups 


457.73 


1 


457.73 


6 . 16* 


Error Between 


3420.86 


46 


74 . 37 




Within Subjects 


109684.33 


96 






tests 


101565,08 


2 


50782.54 


65 . 22* 


tests X Sx w/in 


gps 7163.75 


92 


77.87 




interaction 


955.51 


2 


477.75 


6 . 14* 


Total 


113562.92 


143 







Individual Comparisons; 

Differences between Tj^ - T 2 3 nd T]|^ - T 3 are significant. 
Difference between T 7 ~ T 3 is not significant. 

= 20.50, 12 ''= 77.61, 



13 = 77.35 



*P < .05 



**p< .01 



***p < .001 



t - 
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study 20: Stegner and Stone - Design #3 

Table 20a- Means and Standard Deviations 



Pretest Posttest 



Program 


then 


Lecture 


M 


29.64 


r wo u uco u 

79.89 








SD 


4.76 


7.05 








N 


28 




Lecture 


then 


Program 


M 


31.93 


81.59 








SD 


11.41 


8.09 









N 


29 





Table 20b- Analy sis of Variance 

Source d£ MS 



Between Subjects 
Groups 


1.0 


112.94 


1.39 


Error Between 


54.9 


81.07 


Within Subjects 

Learning (Pre-Post) 


1.0 


71093.08 


1179. 18*** 


Interaction 


1.0 


2.51 


.04 


Tests X Ss w/ in gps 


54.9 


60.29 


Total 


112.9 







Study 21: Wirt - Design #1 

— Table 21a~ Means and Standard Deviation 



Group 




Pretest 


Posttest 


Program 


M 


49.33 


73.17 




SD 


11.39 


8.12 




N 


18 




Lecture 


M 


45.88 


67.25 




SD 


12.61 


10.78 


- 


N 


16 





Table 21b 


- Analysis 


of Variance 




Source 


df 


MS 


F 


Between Subjects 








Groups 


1.0 


372.25 


1.63 


Error Between 


31.8 


228.82 




Within Subjects 








Learning (Pre-Post) 


1.0 


8656.08 


127.54*** 


Interaction 


1.0 


25.60 


.38 


Tests X Ss w/in gps 


31.8 


67.87 


Total 


66.7 






*P < .05 




■*P< .01 


***p < .001 
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-14- 



Study 24: Wust brook - 

Tab 1.0 24a- 


Design #4 
Means of Groups 




Groups 




Pretest 


Posttest 


i: Volimteers - 


M 


45.73 


82.91 


OKtra credit 


N 


11 




2: Volunteers - 


M 


58.27 


72.18 


required credit 


N 


11 




3: Nonvolunteers - 


M 


48.45 


63.09 


OKtra credit 


N 


11 




4: Nonvolunteers - 


M 


53.45 


59.73 


required credit 


N 


11 





Table 24b- 


Analysis 


of Variance 




Source 


df 


MS 


F 


Betvjeen Subjects 
A-Vol. vs Non Vols. 


1 


283.68 


1.79 


B-EKtra vs Req. Cred. 


1 


756.41 


4.76* 


AB- Interact ion 


1 


560.05 


3.53 


Error Between 


40 


158.75 





Within Subjects 
C Learning (Pre-Post) 


1 


3,718.00 


42.81*** 


AC -Interaction 


1 


142.55 


1.64 


BC- In ter act ion 


1 


186.18 


2.14 


ABC -Interact ion 


1 


35.64 


- - - 


Tests X Sk w/ in gps. 


40 


0 




Total 


87 






*P < . 05 




**P< .01 


***p< .001 
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Souhce o# 






Jf 









Be'tioeen 


N-'KCO 


9!p<f-l 
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p-^l 
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9f[(4)-(0] 




AB 
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(p-'Xr-o 
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APPENDIX F«1 



S THE intervibw-scheoule 

Each question (1~9} is based on its correspofiding assumption* 

' X. How do you feel about your present teaching procedures? Nhat 
do you like to dislike about them? Are there any next steps 

j you would like to take in the developjf*«snt of your teaching? 

1 

' 2. In preparing or revising a course • how much attention would you 
give to preparation of content as compared with the methods 
through which the content will be presented. Why? 

I! 

« 

' 3. Briefly, what do you see as the role , or functions and respon- 
' sibilities a) of the teacher? b) of the student? (If present- 
ing the students with questions and problems or giving oppor- 
, tunities for qu«>.stion8 and discussions or interaction is men- 

' tioned, then ask: Just how is this done? for what purposes?) 

i 

‘4. Do course objectives need to be expressly formulated or not? 

I (If interviewee says they need to be so formulated, then ask: 

■ how would they be formulated? how specifically? In what 

j terms? Can you give An example of one or two of your objec- 

tives?) 

js. Is there any way of knowing whether students attain these ob- 
j jectives? (If yes, how do you know?) 

i6. Is there any need for the teacher to analyge and organize 

course material before presenting it to students? (If yes, in 
what specific ways? What principles, if any, of organization 
do ycj feel should be followed? If principles are mentioned 
then ask? Can you give one or two Illustrations of how you 
would apply these principles?) 

7. Is there any need for taking into account students^ previous 
learning - how much they already know before entering your 
courses or a particular session of a c^'surael' (If yes, how 
would you find out what the student knows and takes into 
account? ) 

8. What find of relationship do you favor for yourself between 
students and teachers in the classroom? 

9. What do you think are the major sources of motivation for the 

j students in your classes. What causes them to work? Do you 

I find it necessary or desirable to motivate students? (If so, 

! how?) 




Appendix F«1 (Cont) 



10- When and how frequently do you assess or evaluate the know- 
ledge • understandings, etc . students have gained in your 
courses? (Every session, weekly, mid-term, end of year?) 
Why? What is the purpose of these evaluations? What use is 
made of them? 

11. What are your impressions of the new instructional media and 
of their uses? Which, if any, do you use? Would like to 
use? 



12. Which, if any, of the answers you have given in this interview 
represent very recent changes in your thinking? 

13. Do you happen to be familiar with the methods used in the 
development of programed instruction? If yes, do you feel tha't 
the principles and techniques involved in the preparation of 
programed ir^struction have, or do not have, application to 
other methods of instruction? In other words ^ are the prin- 
ciples and techniques of programing applicable only to pre- 
paring programs or do they have broader applicability in ed- 
ucation? If yes, just how are they applicable? To what 
specific methods of teaching? (The judges were not given the 
answers to this question). 
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TABULATION OF JUDGES RATINGS- JUDGE 
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APPENDIX F-3 



AVERAGE INTERCORRELATION AMONG JUDGES 



S= r2 - 



( R) 2 

N 



w=12 5 

k2n(n2 -1) 



= 41,661.5 " 32,266,6 = 9,394.9 



W = M 1aI38 ._8 _ 
124,200.0 " 



R « KW-1 = 2.724-1 = 1.724 = .862 
K-1 2 2 



= k(N-l)W = 62.652, p .001 
2 .001^23) = 49.73 



APPENDIX F-4 



DIFFERENCE BETWEEN PROGRAMERS 6 NON-PROGRAMERS 



text 



Prog t 



Above Mdn 



Non-Prog : 
3 



12 



Below Mdn 



12 



N 



12 



12 



24 



2 . 



N( AD-BC -2)^ I df 

lATBy(c+D)(AtC ) ( B+D ) 



= (k-D(r-l) 



2 . 



24 C ,81-91 -12) 



24(3600) 86400 



= 12 X 12 X 12 X 12 



20,736 = 20736 



= 4.17 p .05 
^ .05^^^ = 3.8 



APPENDIX F-5 



PROGRAMERS VS. UNINFORMED CONTROLS 



Mann-Whltney 



U-Test 



Working with the sum of ranks 



J2r.S. 
























t: 















139.5 



12 



Sum of ranks programers = 



‘1 = 



Sum of ranks uninformed controls R2 s 31,5 U 2 

Pj CnrH) 

U = nji^nj + 2 • Ri 



= 6 



«150 - 139.5 = 10.5 



p . 01 



U ~ 11. 
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APPENDIX F-7 



There wa® no wa^f in which to analyze the data by means of 
existing teste» so we devised our own method of analysis. 

1. Basic assumptionss 

Lot A and B be two p x q matrices composed of aij and bij ele« 
ments respectively where 

i"l|2|3...p * 

j ~ li2|3...q 

By means of matrix subtraction, a difference matrix, Di con- 
sisting of dij » aij - bij elements may be obtained. 

If the elements of A and B are random variates, then it 
follows that 



dij = 0 



or 

E (d) = o 

According to the central limits theorem, the sampling distri- 
bution of d is expected to be normal. 

Assuming that the D matrix is obtained from A and B which are 
composed of random variates, it is possible to contrast the proper- 
ties of D with those of another difference matrix, DJ consisting of 
dij = aij - fcij elements of A* and B* should be the same 

size. The sampling distribution of d* is expected to be normal* 

If the distributions are normal, then the distribution of the 
difference scores between the two distributions are also normally 
distributed, therefore, the hypothesis that the two distributions^ 

D and D* are drawn from the same population may be tested by the 
ratio; 



z = d/sd. 



2. Procedure 



a) computation of d* matrix 
dij = Plj - cij . 

_ - JErX 












ERIC 
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2. b) Analysis of d* by Judges (Table 2) shows that the judges 

differ with respect to discrimination - for examj^let judge 
B has a higher discrimination index than judge A or C* 
However, the reliability coefficient, r = .78, indicates a 
high degree of concordance among the judges. 



c) The differences between the informed and uninformed control 
groups are not significant, £ (2.33) = .15, p .05. 

d) The differences between the items (Table 3) are significant, 
£ (11,24) = 4.42, p .01. The last column on Table! gives 
the rank order of the items in terms of their discriminabil- 
ity . 

e) The ratio d/sd - .4B9/.4 = 1.22 (Table 1) indicates that the 
D matrix as a whole is a useful device. The probability is 
.90 that the observed D* matrix is different from an ex- 
pected D matrix. 



Table 1 

Matrix and Summary Statistics 



Judges 

Items 


A 


B 


C 


k 

di 


k 2 
di 


di 


Item 

Rank 


1 


.16 


.33 


1.08 


1.57 


1.30009 


. 5233 


7 


2 


.'58 


1.16 


. 67 


2.41 


2.1309 


.8033 


2 


3 


.75 


. 50 


. 50 


1.75 


1.0625 


. 5833 


5 


4 


.42 


.58 


.41 


1.41 


.6809 


.4700 


9 


S 


.16 


.67 


.17 


1.00 


. 5034 


.3333 


10 


6 


1.33 


' .25 


.25 


1.83 


1.8939 


.6099 


4 


7 


-.42 


-.34 


-.34 


-1.10 


.4076 


- . 3666 


12 


8 


.75 


. 59 


.34 


1.68 


1. 0262 


. 5599 


6 


9 


.08 


.66 


.74 


1.48 


. 9896 


.4933 


8 


0 


. 66 


-.25 


. 58 


.99 


.8345 


. 3300 


11 


1 


.67 


.92 


.83 


2.42 


1.9842 


.8066 


• 1 


2 


150 


1.16 


.50 


2.16 


1.8456 


.7199 


3 





































er|c 



dj 5.64 



6.23 



5.73 



17.60 



r 2 

dj 4.7752 5.7061 4.1789 



14.6602 



dj 



4700 .5191 .4775 

( dj)^= 35.3974 

S-i2 s 



.489 



d" ~ 0.16 
Sd = 0.4 







Table 


2 






i 




Analysis of d* 


by 


Judges 




V, 

\ 

> 

t 


Source of variation 


SS 


df 


MS 


F 


1 

\ 


Between Items 




5.97 


11 


.54 




\ 

i. 

\ 


Within Icems 




2.86 


24 


.12 


*** 


\ 


Between Judges 




2.27 


2 


1.14 


38.00 




Residual 




• 59 


22 


.03 




\ 

\ 


Total 




8.83 


35 






) 



*** p .001 



HSw/ltems 
r = 1 - MSb/ items 



= 1 “ .22 

= .78 



Table 3 



Analysis of d* by Items 



Source of Variation 
Between Items 
Error 



'• Tot-al- 



SS 

5.83 
2.86 
-fl- y 6' 9- 



df 

11 

24 

- 3 ^ 



MS 
. 53 
.12 



4.42 ** 



** 



p .01 
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